Efficient context model computation design in transform coefficient coding

ABSTRACT

A processor is configured to maintain, for encoding current values related to the transform coefficients a first line buffer and a second line buffer. The current values are arranged along a current scan-order anti-diagonal line. The first line buffer includes first values of a first scan-order anti-diagonal line. The second line buffer includes second values of a second scan-order anti-diagonal line. The processor is further configured to interleave the first values and the second values in a destination buffer; select, using the destination buffer, a probability distribution for coding a current value of the current values; entropy encode, in a compressed bitstream, the current value using the probability distribution; and replace, for coding values of an immediately subsequent scan-order anti-diagonal line to the current scan-order anti-diagonal line, one of the second line buffer or the first line buffer with the current scan-order anti-diagonal line.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.16/693,438, filed Nov. 25, 2019, which is a continuation in part of U.S.patent application Ser. No. 15/883,323, filed Jan. 30, 2018, the entiredisclosures of which are hereby incorporated by reference.

BACKGROUND

Digital video streams may represent video using a sequence of frames orstill images. Digital video can be used for various applicationsincluding, for example, video conferencing, high definition videoentertainment, video advertisements, or sharing of user-generatedvideos. A digital video stream can contain a large amount of data andconsume a significant amount of computing or communication resources ofa computing device for processing, transmission, or storage of the videodata. Various approaches have been proposed to reduce the amount of datain video streams, including compression and other encoding techniques.

SUMMARY

A first aspect is an apparatus for encoding a transform block oftransform coefficients. The apparatus includes a processor that isconfigured to maintain, for encoding current values related to thetransform coefficients a first line buffer and a second line buffer. Thecurrent values are arranged along a current scan-order anti-diagonalline of a scan order. The first line buffer includes first values of afirst scan-order anti-diagonal line scanned immediately before thecurrent scan-order anti-diagonal line. The second line buffer includessecond values of a second scan-order anti-diagonal line scannedimmediately before the first scan-order anti-diagonal line. Theprocessor is further configured to interleave the first values of thefirst line buffer and the second values of the second line buffer in adestination buffer; select, using the destination buffer, a probabilitydistribution for coding a current value of the current values; entropyencode, in a compressed bitstream, the current value using theprobability distribution; and replace, for coding values of animmediately subsequent scan-order anti-diagonal line to the currentscan-order anti-diagonal line, one of the second line buffer or thefirst line buffer with the current scan-order anti-diagonal line.

A second aspect is an apparatus for decoding a transform block. Theapparatus includes a processor that is configured to store, in a firstline buffer, first values of a first scan-order diagonal line scannedimmediately before a current scan-order diagonal line of the transformblock; store, in a second line buffer, second values of a secondscan-order diagonal line scanned immediately before the first scan-orderdiagonal line; interleave the first values of the first line buffer andthe second values of the second line buffer in a destination buffer;select, using the destination buffer, a probability distribution forcoding a current value of the current scan-order diagonal line; entropydecode, from a compressed bitstream, the current value using theprobability distribution; and replace, for coding values of animmediately subsequent scan-order diagonal line to the currentscan-order diagonal line, one of the second line buffer or the firstline buffer with current values of the current scan-order diagonal line.

A third aspect is a method for decoding a transform block. The methodincludes storing, in a first line buffer, first values of a firstscan-order diagonal line scanned immediately before a current scan-orderdiagonal line of the transform block; storing, in a second line buffer,second values of a second scan-order diagonal line scanned immediatelybefore the first scan-order diagonal line; interleaving the first valuesof the first line buffer and the second values of the second line bufferin a destination buffer; selecting, using the destination buffer, aprobability distribution for coding a current value of the currentscan-order diagonal line; entropy decoding, from a compressed bitstream,the current value using the probability distribution; and replacing, forcoding values of an immediately subsequent scan-order diagonal line tothe current scan-order diagonal line, one of the second line buffer orthe first line buffer with current values of the current scan-orderdiagonal line.

These and other aspects of the present disclosure are disclosed in thefollowing detailed description of the embodiments, the appended claimsand the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The description herein makes reference to the accompanying drawingswherein like reference numerals refer to like parts throughout theseveral views.

FIG. 1 is a schematic of a video encoding and decoding system.

FIG. 2 is a block diagram of an example of a computing device that canimplement a transmitting station or a receiving station.

FIG. 3 is a diagram of a video stream to be encoded and subsequentlydecoded.

FIG. 4 is a block diagram of an encoder according to implementations ofthis disclosure.

FIG. 5 is a block diagram of a decoder according to implementations ofthis disclosure.

FIG. 6 is a flowchart diagram of a process for encoding a transformblock in an encoded video bitstream using level maps according to animplementation of this disclosure.

FIG. 7 is a diagram illustrating the stages of transform coefficientcoding using level maps in accordance with implementations of thisdisclosure.

FIG. 8 is a diagram of previously coded neighbors in a non-zero mapaccording to an implementation of this disclosure.

FIG. 9 is a flowchart diagram of a process for coding a transform blockusing level maps according to an implementation of this disclosure.

FIGS. 10A-B is a diagram of examples of templates for determining acoding context according to implementations of this disclosure.

FIG. 11 is a flowchart diagram of a process for coding a transform blockusing level maps according to an implementation of this disclosure.

FIG. 12 is an example of an illustration of using a template dodetermine a context according to implementations of this disclosure.

FIG. 13 is a diagram of an example of a scan order that aligns valuealong anti-diagonal lines according the implementations of thisdisclosure.

FIG. 14 is flowchart diagram of a process for cache management forcontext selection according to implementations of this disclosure.

FIG. 15 is flowchart diagram of a process for using a moving window forcontext selection according to implementations of this disclosure.

FIG. 16 is flowchart diagram of a process for coding a transform blockof transform coefficients according to an implementation of thisdisclosure.

DETAILED DESCRIPTION

As mentioned above, compression schemes related to coding video streamsmay include breaking images into blocks and generating a digital videooutput bitstream (i.e., an encoded bitstream) using one or moretechniques to limit the information included in the output bitstream. Areceived bitstream can be decoded to re-create the blocks and the sourceimages from the limited information. Encoding a video stream, or aportion thereof, such as a frame or a block, can include using temporalor spatial similarities in the video stream to improve codingefficiency. For example, a current block of a video stream may beencoded based on identifying a difference (residual) between thepreviously coded pixel values, or between a combination of previouslycoded pixel values, and those in the current block.

Encoding using spatial similarities can be known as intra prediction.Intra prediction attempts to predict the pixel values of a block of aframe of video using pixels peripheral to the block; that is, usingpixels that are in the same frame as the block but that are outside theblock. A prediction block resulting from intra prediction is referred toherein as an intra predictor. Intra prediction can be performed along adirection of prediction where each direction can correspond to an intraprediction mode. The intra prediction mode can be signalled by anencoder to a decoder.

Encoding using temporal similarities can be known as inter prediction.Inter prediction attempts to predict the pixel values of a block using apossibly displaced block or blocks from a temporally nearby frame (i.e.,reference frame) or frames. A temporally nearby frame is a frame thatappears earlier or later in time in the video stream than the frame ofthe block being encoded. A prediction block resulting from interprediction is referred to herein as inter predictor.

Inter prediction is performed using a motion vector. A motion vectorused to generate a prediction block refers to a frame other than acurrent frame, i.e., a reference frame. Reference frames can be locatedbefore or after the current frame in the sequence of the video stream.Some codecs use up to eight reference frames, which can be stored in aframe buffer. The motion vector can refer to (i.e., use) one of thereference frames of the frame buffer. As such, one or more referenceframes can be available for coding a current frame.

As mentioned above, a current block of a video stream may be encodedbased on identifying a difference (residual) between the previouslycoded pixel values and those in the current block. In this way, only theresidual and parameters used to generate the residual need be added tothe encoded bitstream. The residual may be encoded using a lossyquantization step.

The residual block can be in the pixel domain. The residual block can betransformed into the frequency domain resulting in a transform block oftransform coefficients. The transform coefficients can be quantizedresulting into a quantized transform block of quantized transformcoefficients. The quantized coefficients can be entropy encoded andadded to an encoded bitstream. A decoder can receive the encodedbitstream, entropy decode the quantized transform coefficients toreconstruct the original video frame.

Entropy coding is a technique for “lossless” coding that relies uponprobability models that model the distribution of values occurring in anencoded video bitstream. By using probability models based on a measuredor estimated distribution of values, entropy coding can reduce thenumber of bits required to represent video data close to a theoreticalminimum. In practice, the actual reduction in the number of bitsrequired to represent video data can be a function of the accuracy ofthe probability model, the number of bits over which the coding isperformed, and the computational accuracy of fixed-point arithmetic usedto perform the coding.

In an encoded video bitstream, many of the bits are used for one of twothings: either content prediction (e.g., inter mode/motion vectorcoding, intra prediction mode coding, etc.) or residual coding (e.g.,transform coefficients).

Encoders may use techniques to decrease the amount of bits spent ontransform coefficient coding. For example, level maps, as furtherdescribed below, can be used for coding transform coefficients. Herein,for brevity and unless otherwise clear from the context, transformcoefficient(s) means quantized transform coefficient(s), and transformblock(s) means quantized transform coefficient(s).

The values of the transform coefficients can be split into two planes: alower plane and a higher plane. A maximum level is associated with thelower plane. The lower plane indicates for a transform coefficient avalue that is “up to whether the coefficient is less than or equal themaximum level.” For example, if the maximum level is 2, then the lowerplane indicates whether a coefficient is 0, 1, 2, or greater than 2. Assuch, “up to whether the coefficient is less than or equal 2” means thatthe lower plane indicates that the value of a coefficient is either 0,1, 2, or greater than 2. Any coefficient whose value is greater than 2(e.g., the maximum level) can be represented in lower plane with onevalue that indicates “greater than 2” (i.e., 3 or above). The higherplane corresponds to coefficient values that are, in this example, 3 orabove. As such, each of the values of the lower plane can be one of fourvalues, namely 0, 1, 2, or “greater than 2.” In an example, acoefficient that is greater than 2 can be represented by the value 3 inthe lower plane.

In general, the lower plane (i.e., values of the lower plane) appearsmuch more frequently than the higher plane. This is so because for everyvalue of higher plane, a corresponding value necessary exists in thelower plane. However, some of the values of the lower plane may not havecorresponding values in the higher plane. For example, a transformcoefficient with a value of 7 would have corresponding values in thelower plane (e.g., a value of 3 since 7 is greater than 2) and a valuein the lower plane (e.g., 5). However, a transform coefficient with avalue of 2 does not have a corresponding value in the higher map. Assuch, the compression performance (i.e., speed of compression) of thelower plane can have a significant impact on the overall compressionperformance.

As described above, entropy coding a sequence of symbols is typicallyachieved by using a probability model to determine a probability p forthe sequence and then using binary arithmetic coding to map the sequenceto a binary codeword at the encoder and to decode that sequence from thebinary codeword at the decoder. The length (i.e., number of bits) of thecodeword is given by -log(p). The efficiency of entropy coding can bedirectly related to the probability model. Throughout this document, logdenotes the logarithm function to base two (2) unless specifiedotherwise.

A model, as used herein, can be, or can be a parameter in, a lossless(entropy) coding. A model can be any parameter or method that affectsprobability estimation for entropy coding.

A purpose of context modeling is to obtain probability distributions fora subsequent entropy coding engine, such as arithmetic coding, Huffmancoding, and other variable-length-to-variable-length coding engines. Toachieve good compression performance, a large number of context modelsmay be required. For example, some video coding systems can includehundreds or even thousands of context models for transform coefficientcoding alone. Each context model can correspond to a probabilitydistribution.

As already mentioned, residuals for a block of video are transformedinto transform blocks of transform coefficients. The transform blocksare in the frequency domain and one or more transform blocks may begenerated for a block of video. The transform coefficients are quantizedand entropy coded into an encoded video bitstream. A decoder uses theencoded transform coefficients and the reference frames to reconstructthe block. Entropy coding a transform coefficient involves the selectionof a context model (also referred to as probability context model orprobability model) which provides estimates of conditional probabilitiesfor coding the binary symbols of a binarized transform coefficient.

The available context models in a codec may be available as a list, anarray, a file, or some other suitable data structure. The list ofcontext models can be ordered. That is, a respective context index canbe associated with each context model. As such, selecting a contextmodel can involve determining a context index. The context index is thenused to select (e.g., retrieve, access, etc.) the corresponding contextmodel.

Coding the transform coefficients of a transform block, such as codingthe values of the lower plane corresponding to the transform block, caninclude visiting each value of the lower plane, determining a contextindex for the value of the lower plane, retrieving the context modelassociated with the context index, and entropy coding the value usingthe context model.

The values of the lower plane are visited (e.g., processed) along a scanorder. The scan order can also be referred to as a processing order.Visiting the values of the lower plane along a scan order can includegenerating a one-dimensional array. Determining a context index, for acurrent transform coefficient of a transform block, can includeaccessing other values in the one-dimensional array. These other valuesare values that correspond to other transform coefficients that neighborthe current transform coefficients in the 2-dimensional transform block.While these other transform coefficients may neighbor the currenttransform coefficient block in the 2-dimensional block, thecorresponding values may not be neighboring values in theone-dimensional array. The arrangement of the values of the lower planein the on-dimensional array depends on the scan order.

As further explained below with respect to FIG. 12, accessing therespective other values for each transform coefficient can imposecertain computational barrier, especially for the decoder speed.

Implementations of this disclosure can result in improved compressionperformance by arranging and accessing values of the lower plane thatare required for determining context indexes in such ways as to optimizememory access. The values of the lower plane can be arranged alonganti-diagonal lines of a scan order in one or more line buffers suchthat determining a context index for a value of the lower plane accessescontiguous locations in two or more line buffers. A scan order thataligns values along anti-diagonal lines and methods for accessing thevalues from line buffers are described.

By integrating the scan order (e.g., processing order) of the values ofthe lower plane and the computation method of a context index, access tonon-contiguous locations of line buffers can be reduced and parallelcomputing can be exploited. As such, implementations of this disclosurecan have reduced computational.

Efficient context model computation design in transform coefficientcoding is described herein first with reference to a system in which theteachings may be incorporated.

FIG. 1 is a schematic of a video encoding and decoding system 100. Atransmitting station 102 can be, for example, a computer having aninternal configuration of hardware such as that described in FIG. 2.However, other suitable implementations of the transmitting station 102are possible. For example, the processing of the transmitting station102 can be distributed among multiple devices.

A network 104 can connect the transmitting station 102 and a receivingstation 106 for encoding and decoding of the video stream. Specifically,the video stream can be encoded in the transmitting station 102 and theencoded video stream can be decoded in the receiving station 106. Thenetwork 104 can be, for example, the Internet. The network 104 can alsobe a local area network (LAN), wide area network (WAN), virtual privatenetwork (VPN), cellular telephone network or any other means oftransferring the video stream from the transmitting station 102 to, inthis example, the receiving station 106.

The receiving station 106, in one example, can be a computer having aninternal configuration of hardware such as that described in FIG. 2.However, other suitable implementations of the receiving station 106 arepossible. For example, the processing of the receiving station 106 canbe distributed among multiple devices.

Other implementations of the video encoding and decoding system 100 arepossible. For example, an implementation can omit the network 104. Inanother implementation, a video stream can be encoded and then storedfor transmission at a later time to the receiving station 106 or anyother device having memory. In one implementation, the receiving station106 receives (e.g., via the network 104, a computer bus, and/or somecommunication pathway) the encoded video stream and stores the videostream for later decoding. In an example implementation, a real-timetransport protocol (RTP) is used for transmission of the encoded videoover the network 104. In another implementation, a transport protocolother than RTP may be used, e.g., an HTTP-based video streamingprotocol.

When used in a video conferencing system, for example, the transmittingstation 102 and/or the receiving station 106 may include the ability toboth encode and decode a video stream as described below. For example,the receiving station 106 could be a video conference participant whoreceives an encoded video bitstream from a video conference server(e.g., the transmitting station 102) to decode and view and furtherencodes and transmits its own video bitstream to the video conferenceserver for decoding and viewing by other participants.

FIG. 2 is a block diagram of an example of a computing device 200 thatcan implement a transmitting station or a receiving station. Forexample, the computing device 200 can implement one or both of thetransmitting station 102 and the receiving station 106 of FIG. 1. Thecomputing device 200 can be in the form of a computing system includingmultiple computing devices, or in the form of a single computing device,for example, a mobile phone, a tablet computer, a laptop computer, anotebook computer, a desktop computer, and the like.

A CPU 202 in the computing device 200 can be a central processing unit.Alternatively, the CPU 202 can be any other type of device, or multipledevices, capable of manipulating or processing information now-existingor hereafter developed. Although the disclosed implementations can bepracticed with a single processor as shown, e.g., the CPU 202,advantages in speed and efficiency can be achieved using more than oneprocessor.

A memory 204 in the computing device 200 can be a read-only memory (ROM)device or a random access memory (RAM) device in an implementation. Anyother suitable type of storage device can be used as the memory 204. Thememory 204 can include code and data 206 that is accessed by the CPU 202using a bus 212. The memory 204 can further include an operating system208 and application programs 210, the application programs 210 includingat least one program that permits the CPU 202 to perform the methodsdescribed here. For example, the application programs 210 can includeapplications 1 through N, which further include a video codingapplication that performs the methods described here. The computingdevice 200 can also include a secondary storage 214, which can, forexample, be a memory card used with a computing device 200 that ismobile. Because the video communication sessions may contain asignificant amount of information, they can be stored in whole or inpart in the secondary storage 214 and loaded into the memory 204 asneeded for processing.

The computing device 200 can also include one or more output devices,such as a display 218. The display 218 may be, in one example, a touchsensitive display that combines a display with a touch sensitive elementthat is operable to sense touch inputs. The display 218 can be coupledto the CPU 202 via the bus 212. Other output devices that permit a userto program or otherwise use the computing device 200 can be provided inaddition to or as an alternative to the display 218. When the outputdevice is or includes a display, the display can be implemented invarious ways, including by a liquid crystal display (LCD), a cathode-raytube (CRT) display or light emitting diode (LED) display, such as anorganic LED (OLED) display.

The computing device 200 can also include or be in communication with animage-sensing device 220, for example a camera, or any otherimage-sensing device 220 now existing or hereafter developed that cansense an image such as the image of a user operating the computingdevice 200. The image-sensing device 220 can be positioned such that itis directed toward the user operating the computing device 200. In anexample, the position and optical axis of the image-sensing device 220can be configured such that the field of vision includes an area that isdirectly adjacent to the display 218 and from which the display 218 isvisible.

The computing device 200 can also include or be in communication with asound-sensing device 222, for example a microphone, or any othersound-sensing device now existing or hereafter developed that can sensesounds near the computing device 200. The sound-sensing device 222 canbe positioned such that it is directed toward the user operating thecomputing device 200 and can be configured to receive sounds, forexample, speech or other utterances, made by the user while the useroperates the computing device 200.

Although FIG. 2 depicts the CPU 202 and the memory 204 of the computingdevice 200 as being integrated into a single unit, other configurationscan be utilized. The operations of the CPU 202 can be distributed acrossmultiple machines (each machine having one or more of processors) thatcan be coupled directly or across a local area or other network. Thememory 204 can be distributed across multiple machines such as anetwork-based memory or memory in multiple machines performing theoperations of the computing device 200. Although depicted here as asingle bus, the bus 212 of the computing device 200 can be composed ofmultiple buses. Further, the secondary storage 214 can be directlycoupled to the other components of the computing device 200 or can beaccessed via a network and can comprise a single integrated unit such asa memory card or multiple units such as multiple memory cards. Thecomputing device 200 can thus be implemented in a wide variety ofconfigurations.

FIG. 3 is a diagram of an example of a video stream 300 to be encodedand subsequently decoded. The video stream 300 includes a video sequence302. At the next level, the video sequence 302 includes a number ofadjacent frames 304. While three frames are depicted as the adjacentframes 304, the video sequence 302 can include any number of adjacentframes 304. The adjacent frames 304 can then be further subdivided intoindividual frames, e.g., a frame 306. At the next level, the frame 306can be divided into a series of segments 308 or planes. The segments 308can be subsets of frames that permit parallel processing, for example.The segments 308 can also be subsets of frames that can separate thevideo data into separate colors. For example, the frame 306 of colorvideo data can include a luminance plane and two chrominance planes. Thesegments 308 may be sampled at different resolutions.

Whether or not the frame 306 is divided into the segments 308, the frame306 may be further subdivided into blocks 310, which can contain datacorresponding to, for example, 16×16 pixels in the frame 306. The blocks310 can also be arranged to include data from one or more segments 308of pixel data. The blocks 310 can also be of any other suitable sizesuch as 4×4 pixels, 8×8 pixels, 16×8 pixels, 8×16 pixels, 16×16 pixelsor larger.

FIG. 4 is a block diagram of an encoder 400 in accordance withimplementations of this disclosure. The encoder 400 can be implemented,as described above, in the transmitting station 102 such as by providinga computer software program stored in memory, for example, the memory204. The computer software program can include machine instructionsthat, when executed by a processor such as the CPU 202, cause thetransmitting station 102 to encode video data in manners describedherein. The encoder 400 can also be implemented as specialized hardwareincluded in, for example, the transmitting station 102. The encoder 400has the following stages to perform the various functions in a forwardpath (shown by the solid connection lines) to produce an encoded orcompressed bitstream 420 using the video stream 300 as input: anintra/inter prediction stage 402, a transform stage 404, a quantizationstage 406, and an entropy encoding stage 408. The encoder 400 may alsoinclude a reconstruction path (shown by the dotted connection lines) toreconstruct a frame for encoding of future blocks. In FIG. 4, theencoder 400 has the following stages to perform the various functions inthe reconstruction path: a dequantization stage 410, an inversetransform stage 412, a reconstruction stage 414, and a loop filteringstage 416. Other structural variations of the encoder 400 can be used toencode the video stream 300.

When the video stream 300 is presented for encoding, the frame 306 canbe processed in units of blocks. At the intra/inter prediction stage402, a block can be encoded using intra-frame prediction (also calledintra-prediction) or inter-frame prediction (also calledinter-prediction), or a combination of both. In any case, a predictionblock can be formed. In the case of intra-prediction, all or a part of aprediction block may be formed from samples in the current frame thathave been previously encoded and reconstructed. In the case ofinter-prediction, all or part of a prediction block may be formed fromsamples in one or more previously constructed reference framesdetermined using motion vectors.

Next, still referring to FIG. 4, the prediction block can be subtractedfrom the current block at the intra/inter prediction stage 402 toproduce a residual block (also called a residual). The transform stage404 transforms the residual into transform coefficients in, for example,the frequency domain using block-based transforms. Such block-basedtransforms include, for example, the Discrete Cosine Transform (DCT) andthe Asymmetric Discrete Sine Transform (ADST). Other block-basedtransforms are possible. Further, combinations of different transformsmay be applied to a single residual. In one example of application of atransform, the DCT transforms the residual block into the frequencydomain where the transform coefficient values are based on spatialfrequency. The lowest frequency (DC) coefficient at the top-left of thematrix and the highest frequency coefficient at the bottom-right of thematrix. It is worth noting that the size of a prediction block, andhence the resulting residual block, may be different from the size ofthe transform block. For example, the prediction block may be split intosmaller blocks to which separate transforms are applied.

The quantization stage 406 converts the transform coefficients intodiscrete quantum values, which are referred to as quantized transformcoefficients, using a quantizer value or a quantization level. Forexample, the transform coefficients may be divided by the quantizervalue and truncated. The quantized transform coefficients are thenentropy encoded by the entropy encoding stage 408. Entropy coding may beperformed using any number of techniques, including token and binarytrees. The entropy-encoded coefficients, together with other informationused to decode the block, which may include for example the type ofprediction used, transform type, motion vectors and quantizer value, arethen output to the compressed bitstream 420. The information to decodethe block may be entropy coded into block, frame, slice and/or sectionheaders within the compressed bitstream 420. The compressed bitstream420 can also be referred to as an encoded video stream or encoded videobitstream, and the terms will be used interchangeably herein.

The reconstruction path in FIG. 4 (shown by the dotted connection lines)can be used to ensure that both the encoder 400 and a decoder 500(described below) use the same reference frames and blocks to decode thecompressed bitstream 420. The reconstruction path performs functionsthat are similar to functions that take place during the decodingprocess that are discussed in more detail below, including dequantizingthe quantized transform coefficients at the dequantization stage 410 andinverse transforming the dequantized transform coefficients at theinverse transform stage 412 to produce a derivative residual block (alsocalled a derivative residual). At the reconstruction stage 414, theprediction block that was predicted at the intra/inter prediction stage402 can be added to the derivative residual to create a reconstructedblock. The loop filtering stage 416 can be applied to the reconstructedblock to reduce distortion such as blocking artifacts.

Other variations of the encoder 400 can be used to encode the compressedbitstream 420. For example, a non-transform based encoder 400 canquantize the residual signal directly without the transform stage 404for certain blocks or frames. In another implementation, an encoder 400can have the quantization stage 406 and the dequantization stage 410combined into a single stage.

FIG. 5 is a block diagram of a decoder 500 in accordance withimplementations of this disclosure. The decoder 500 can be implementedin the receiving station 106, for example, by providing a computersoftware program stored in the memory 204. The computer software programcan include machine instructions that, when executed by a processor suchas the CPU 202, cause the receiving station 106 to decode video data inthe manners described below. The decoder 500 can also be implemented inhardware included in, for example, the transmitting station 102 or thereceiving station 106.

The decoder 500, similar to the reconstruction path of the encoder 400discussed above, includes in one example the following stages to performvarious functions to produce an output video stream 516 from thecompressed bitstream 420: an entropy decoding stage 502, adequantization stage 504, an inverse transform stage 506, anintra/inter-prediction stage 508, a reconstruction stage 510, a loopfiltering stage 512 and a post filtering stage 514. Other structuralvariations of the decoder 500 can be used to decode the compressedbitstream 420.

When the compressed bitstream 420 is presented for decoding, the dataelements within the compressed bitstream 420 can be decoded by theentropy decoding stage 502 to produce a set of quantized transformcoefficients. The dequantization stage 504 dequantizes the quantizedtransform coefficients (e.g., by multiplying the quantized transformcoefficients by the quantizer value), and the inverse transform stage506 inverse transforms the dequantized transform coefficients using theselected transform type to produce a derivative residual that can beidentical to that created by the inverse transform stage 412 in theencoder 400. Using header information decoded from the compressedbitstream 420, the decoder 500 can use the intra/inter-prediction stage508 to create the same prediction block as was created in the encoder400, e.g., at the intra/inter prediction stage 402. At thereconstruction stage 510, the prediction block can be added to thederivative residual to create a reconstructed block. The loop filteringstage 512 can be applied to the reconstructed block to reduce blockingartifacts. Other filtering can be applied to the reconstructed block. Inan example, the post filtering stage 514 is applied to the reconstructedblock to reduce blocking distortion, and the result is output as anoutput video stream 516. The output video stream 516 can also bereferred to as a decoded video stream, and the terms will be usedinterchangeably herein.

Other variations of the decoder 500 can be used to decode the compressedbitstream 420. For example, the decoder 500 can produce the output videostream 516 without the post filtering stage 514. In some implementationsof the decoder 500, the post filtering stage 514 is applied after theloop filtering stage 512. The loop filtering stage 512 can include anoptional deblocking filtering stage. Additionally, or alternatively, theencoder 400 includes an optional deblocking filtering stage in the loopfiltering stage 416.

Some codecs may use level maps to code (i.e., encode by an encoder ordecode by a decoder) a transform block. That is, some codecs may uselevel maps to code the transform coefficients of the transform blocks.In level map coding, the transform block is decomposed into multiplelevel maps such that the level maps break down (i.e., reduce) the codingof each transform coefficient value into a series of binary decisionseach corresponding to a magnitude level (i.e., a map level). Thedecomposition can be done by using a multi-run process. As such, atransform coefficient of the transform block is decomposed into a seriesof level binaries and a residue according to the equation:

${{{coefficient}\lbrack r\rbrack}\;\lbrack c\rbrack} = {\left\{ {\left( {\sum\limits_{k = 0}^{T}{{{level}_{k}\lbrack r\rbrack}\;\lbrack c\rbrack}} \right) + {{{residue}\;\lbrack r\rbrack}\;\lbrack c\rbrack}} \right\}*{{{sign}\lbrack r\rbrack}\;\lbrack c\rbrack}}$Where residue[r] [c] = absolute(coefficient[r][c]) − T − 1${{{sign}\lbrack r\rbrack}\;\lbrack c\rbrack} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} {{{coefficient}\;\lbrack r\rbrack}\;\lbrack c\rbrack}} > 0} \\{- 1} & {{{if}\mspace{14mu} {{{coefficient}\;\lbrack r\rbrack}\;\lbrack c\rbrack}} < 0}\end{matrix} \right.$

In the above equation, coefficient[r][c] is the transform coefficient ofthe transform block at the position (row=r, column=c), T is the maximumlevel, level_(k) is the level map corresponding to map level k, residueis a coefficient residual map, and sign is the sign map of the transformcoefficients. These terms are further described below with respect toFIG. 7. The transform coefficients of a transform block can bere-composed using the same equation, such as by a decoder, from encodedlevel_(k) maps, residual map residue, and sign map sign.

A zeroth run can be used to determine a non-zero map (also referred toas a level-0 map) which indicates which transform coefficients of thetransform block are zero and which are non-zero. Level mapscorresponding to runs 1 through a maximum (i.e., threshold) level T(i.e., level-1 map, level-2 map, . . . , level-T map) are generated inascending order from level 1 to the maximum level T. The level map forlevel k, referred to as the level-k map, indicates which transformcoefficients of the transform block have absolute values greater to orequal to k. The level maps are binary maps. A final run generates acoefficients residue map. If the transform block contains transformcoefficient values above the maximum level T, the coefficients residuemap indicates the extent (i.e., residue) that these coefficients aregreater than the maximum level T.

When generating (i.e., coding) the level-k map, only the positions (r,c) corresponding to positions (r, c) of the level-(k-1) map which areequal to 1 (i.e., level_(k-1)[r][c] =1) need be processed—otherpositions of the level-(k-1) are determined to be less than k and,therefore, there is no need to process them for the level-k map. Thisreduces processing complexity and reduces the amount of binary codingoperations.

As the level maps contain binary values, the above and left neighbors ofa value to be encoded are binary values. A context model based on thebinary values of any number of previously coded neighbors can bedetermined. The context model can fully utilize information from allthese neighbors. The previously coded neighbors can be neighbors in thesame level map or a preceding level map, such as an immediatelypreceding level map. The immediately preceding map of the level-k (e.g.,level-2) map is the level-(k-1) (e.g., level-1) map. Contexts accordingto this disclosure can be less complex thereby resulting in efficientmodels for coding the level maps.

When encoding a level-k map, the fully coded level-(k-1) map and thepartially coded level-k map can be used as context information forcontext modeling. As compared to transform coefficient coding of othervideo systems, which code one coefficient value at a time before movingto next transform coefficient, implementations of this disclosure canreduce the cardinality of the reference sample set. This is so because,as further described herein, the information from the level-(k-1) mapand partially coded level-k map are binary information. The binaryinformation enables the use of sophisticated spatial neighboringtemplates for context modeling binary information. Such spatialneighboring templates can better capture statistical characteristics oftransform blocks, especially those with larger transform block sizes.

FIG. 6 is a flowchart diagram of a process 600 for encoding a transformblock in an encoded video bitstream using level maps according to animplementation of this disclosure. The process 600 can be implemented inan encoder such as the encoder 400. The encoded video bitstream can bethe compressed bitstream 420 of FIG. 4.

The process 600 can be implemented, for example, as a software programthat can be executed by computing devices such as transmitting station102. The software program can include machine-readable instructions thatcan be stored in a memory such as the memory 204 or the secondarystorage 214, and that can be executed by a processor, such as CPU 202,to cause the computing device to perform the process 600. In at leastsome implementations, the process 600 can be performed in whole or inpart by the entropy encoding stage 408 of the encoder 400.

The process 600 can be implemented using specialized hardware orfirmware. Some computing devices can have multiple memories, multipleprocessors, or both. The steps or operations of the process 600 can bedistributed using different processors, memories, or both. Use of theterms “processor” or “memory” in the singular encompasses computingdevices that have one processor or one memory as well as devices thathave multiple processors or multiple memories that can be used in theperformance of some or all of the recited steps.

The process 600 is now explained with reference to FIG. 7. FIG. 7 is adiagram illustrating the stages 700 of transform coefficient codingusing level maps in accordance with implementations of this disclosure.FIG. 7 includes the zigzag forward scan order 702, a transform block704, a non-zero map 706, a level-1 map 707, a level-2 map 709, anend-of-block map 726, a sign map 732, and a coefficient residual map734.

The process 600 can receive a transform block, such as the transformblock 704 of FIG. 7. The transform block can be received from thequantization step of an encoder, such as the quantization stage 406 ofthe encoder 400 of FIG. 4. The transform block 704 includes zero andnon-zero transform coefficients. Some of the non-zero coefficients maybe negative values.

At 602, a non-zero map is encoded. The non-zero map indicates positionsof the transform block that contain non-zero transform coefficients. Thenon-zero map can also be referred to as the level-0 map.

The non-zero map 706 of FIG. 6 illustrates a non-zero map. The non-zeromap can be generated by traversing the transform block 704 in a scanorder, such as the zigzag forward scan order 702 of FIG. 7, andindicating in the non-zero map 706, using binary values, whether thecorresponding transform coefficient is a zero or a non-zero. In thenon-zero map 706, a non-zero transform coefficient of the transformblock 704 is indicated with the binary value 1 (one) and a zerotransform coefficient is indicated with the binary value 0 (zero).However, the indication can be reversed (i.e., a zero to indicate anon-zero transform coefficient and one (1) to indicate a zero transformcoefficient).

In an implementation, zero transform coefficients that are beyond (i.e.,come after) the last non-zero transform coefficient, based on the scanorder of the transform block, are not indicated in the non-zero map. Forexample, using the zigzag forward scan order 702 to scan the transformblock 704, the last non-zero transform coefficient 708, corresponding toscan position 11, is the last indicated transform coefficient in thenon-zero map 706 at last non-zero coefficient 710. No values areindicated in the non-zero map 706 for the transform coefficientscorresponding to the scan positions 12-15 of the zigzag forward scanorder 702.

At 604, the process 600 encodes a respective lower-range level map. Eachlower-range map has a map level up to a maximum level. A lower-rangelevel map indicates which values of the non-zero transform coefficientsare equal to the map level of the lower-range map and which values ofthe non-zero transform coefficients are greater than the map level.

For each map level k, up to the maximum level T, a lower-range level maplevel_(k) is encoded. Each lower-range level map indicates which valuesof the transform block are equal to the map level of the lower-rangelevel map and which values of the transform block are greater than themap level. As such, the process 600, using multiple runs (i.e., each runcorresponding to a level k=1, 2, . . . , T), breaks down the coding oftransform coefficients into a series of binary decisions eachcorresponding to a magnitude level. The binary decision of a coefficientat row and column (r, c) in the transform block at level k can bedefined by:

$\begin{matrix}{{{{level}_{k}\lbrack r\rbrack}\;\lbrack c\rbrack} = 1} & {{{{if}\mspace{14mu} {{absolute}\left( {{{coefficient}\lbrack r\rbrack}\;\lbrack c\rbrack} \right)}} > k}} \\{= 0} & {{{{if}\mspace{14mu} {{absolute}\left( {{{coefficient}\lbrack r\rbrack}\;\lbrack c\rbrack} \right)}} \leq k}}\end{matrix}$

For example, for k=1 (i.e., for the level-1 map 707), the process 600determines for each transform coefficient of the transform block 704whether the absolute value of the transform coefficient is greater thank (i.e., 1) or less than or equal to k. For the transform coefficient720 (i.e., at r=0, c=0), as the absolute value of −7 (i.e., |−7 |=7) isgreater than 1, the process 600 sets the corresponding value 722 of thelevel-1 map 707 to 1. For the last non-zero transform coefficient 708(i.e., at r=2, c=2), as the absolute value of −1 (i.e., |−1 |=1) isequal to k (i.e., 1), the process 600 sets the corresponding value 716of the level-1 map 707 to 0. The last non-zero transform coefficient inthe transform block (e.g., the last non-zero transform coefficient 708)can be referred to as the highest AC coefficient).

In an implementation, to generate a lower plane, the process 600 canscan the preceding level map backwards starting at the last 1 value ofthe previous level map. For a level-k map, the preceding level map isthe level-(k-1) map corresponding to the preceding map level (k-1). Thatis, for k=2, the preceding level map is the level-1 map. For k=1, thepreceding level map is the level-0 map (i.e., the non-zero map). For thelevel-1 map 707, scanning of the non-zero map 706 starts at the lastnon-zero coefficient 710. For the level-2 map 709, scanning of thelevel-1 map 707 starts at the last non-zero coefficient 724. Ingenerating a level-k map, the process 600 need only process thetransform coefficients corresponding to 1 values in the level-(k-1). Theprocess 600 need not process the transform coefficients corresponding tonon 1 values as those values are already determined to either be equalto k-1 (i.e., the zero values of the level-(k-1) map) or are less thank-1 (i.e., the blank values of the level-(k-1) map).

In an implementation, the maximum level T can be fixed. For example, themaximum level T can be provided as a configuration to the process 600,the maximum level T can be hard-coded in a program that implements theprocess 600, or the maximum level T can be set statistically oradaptively based on previously coded transform blocks or other blocks ofthe encoded video bitstream. Alternatively, the maximum level T isdetermined by the process 600. That is, the process 600 can testdifferent values for the maximum level T (i.e., T=1, 2, 3, 4, . . . )and determine which value provides the best compression performance. Thevalue of the maximum level T that results in the best compression can beencoded in the video bitstream, which a decoder, such as the decoder 500of FIG. 5 can decode and use. A maximum level T of 2 or 3 has beendetermined to provide acceptable compression as compared to other valuesfor the maximum level T.

At 606, the process 600 encodes a coefficient residual map. Eachresidual coefficient of the coefficient residual map corresponds to arespective (i.e., co-located) non-zero transform coefficient of thetransform block having an absolute value exceeding the maximum level.The residual coefficient for a transform coefficient at location (r, c)of the transform block can be calculated using the formula (1):

residue[r][c]=absolute(coefficient[r][c])−T−1  (1)

FIG. 7 illustrates a coefficients residue map 734. In the example ofFIG. 7, the maximum level T is equal to two (2). As such, thecoefficients residue map 734 contains the residuals of the transformcoefficients of the transform block 704 the absolute values of which aregreater than 2. A residual coefficient is the extent to which theabsolute value of a transform coefficient exceeds the maximum level T.The absolute values of two values of the transform block 704 are greaterthan the value of the maximum level T (i.e., 2), namely the transformcoefficient 720 (i.e., |−7 |=7>2) and transform coefficient 739 (i.e.,|4 |=4>2). Respectively, the coefficients residue map 734 includesresidual 736 and residual 738. Using the formula (1), the residual 736is set to 5 (i.e., absolute(−7)−3=4) and the residual 738 is set to 1(i.e., absolute(4)−3=1).

The residual coefficients of the coefficients residue map 734 can beencoded in the encoded video bitstream using binary coding. Aprobability distribution that fits the statistics of the residualcoefficients of the coefficients residue map can be used. Theprobability distribution can be a geometric distribution, a Laplaciandistribution, a Pareto distribution, or any other distribution.

Encoding the residual coefficients in the encoded video bitstreamprovides several benefits, such as over video coding systems that encodethe transform coefficients. As each residual coefficient is smaller inmagnitude than its corresponding transform coefficient, less bits arerequired to encode the residual coefficient. Additionally, as there arefewer residual coefficients to encode (e.g., 2 in the coefficientresidual map 734 of FIG. 7) than non-zero transform coefficients (e.g.,7 in the transform block 704 of FIG. 7), additional compression canresult.

In an implementation of the process 600, a sign map can also be encoded.A sign map indicates which transform coefficients of the transform blockhave positive values and which transform coefficients have negativevalues. Transform coefficients that are zero need not be indicated inthe sign map. The sign map 732 of FIG. 7 illustrates an example of asign map for the transform block 704. In the sign map, negativetransform coefficients are indicated with a −1 and positive transformcoefficients are indicated with a 1. In some implementations, the signof a positive coefficient may be indicated with a 0 and the sign of anegative coefficient may be indicated with a 1.

In an implementation of the process 600, encoding a non-zero map, at602, can also include generating an end-of-block map for the transformblock and interleaving the non-zero map and the end-of-block map in theencoded video bitstream.

The end-of-block map indicates whether a non-zero transform coefficientof the transform block is the last non-zero coefficient with respect toa given scan order. If a non-zero coefficient is not the last non-zerocoefficient in the transform block, then it can be indicated with thebinary value 0 (zero) in the end-of-block map. If, on the other hand, anon-zero coefficient is the last non-zero coefficient in the transformblock, then it can be indicated with the binary value 1 (one) in theend-of-block map.

For example, as the transform coefficient 720 of the transform block 704is followed by another non-zero transform coefficient (e.g., thetransform coefficient −1 corresponding to scan location 2), thetransform coefficient 720 is not the last non-zero transformcoefficient, it is indicated with the end-of-block value 728 of zero. Onthe other hand, as the transform coefficient corresponding to the scanlocation 11 (i.e., the last non-zero transform coefficient 708) is thelast non-zero coefficient of the transform block 704, it is indicatedwith the end-of-block value 730 of 1 (one).

The process 600 can, by traversing the non-zero map and the end-of-blockmaps in a same scan order, interleave values from the non-zero map 706and the end-of-block map 726 in the encoded bitstream. The process 600can use the zigzag forward scan order 702 or any arbitrary scan order.For each position (r, c), the value at that row and column of thenon-zero map 706 (i.e., nz_map[r][c]) is coded first. If the valuenz_map[r][c] is 1, then the corresponding value from the end-of-blockmap 726 (i.e., eob_map[r][c]) is coded next to indicate whether theposition (r, c) of the transform block 704 contains the last nonzerotransform coefficient. The process 600 ends the coding of the non-zeromap (e.g., the non-zero map 706) when eob_map[r][c] equals to 1 or whenthe last position in the transform block (e.g., the scan position 15 ofthe zigzag forward scan order 702) is reached. That is, when encoding avalue of 1 from the non-zero map 706, the value is followed by anothersyntax element (i.e., a value to be encoded in the encoded videobitstream) from a corresponding (i.e., co-located) end-of-block map 726value to indicate whether the 1 value is the last 1 value of thenon-zero map 706.

In an implementation, encoding a non-zero map, at 602, can also includedetermining a coding context for a value (i.e., to-be-coded value) ofthe non-zero map. The coding context of a to-be-coded value at a currentposition (r, c) can be based on previously coded non-zero neighboringvalues of the to-be-coded value in the non-zero map. The coding contextcan also be based on the position of the to-be-coded value within thenon-zero map.

As mentioned above, the context information can be determined based onthe number of non-zero previously coded neighbors of the currentposition and can be calculated using the sum

non_zero_map_sum(r, c)=Σ_((r′,c′)ϵnb(r,c)) nz_map(r′, c′)  (2)

In equation (2), non_zero_map_sum(r, c) is the number of non-zeropreviously coded neighbors of the to-be-coded value of the non-zeroblock at position (r, c), nb(r,c) is the set of previously codedneighbors of the to-be-coded value at location (r,c) of the non-zeromap, and nz_map(r′, c′) is the value at position (r′, c′) in thenon-zero map. Equation (1) is further explained with reference to FIG.8.

FIG. 8 is a diagram of previously coded neighbors in a non-zero map 800according to an implementation of this disclosure. FIG. 8 includes ato-be-encoded value, current value 802, an unavailable context neighbor806 (i.e., a neighboring value for which context information is notavailable), and coded context neighbors, such as coded context neighbor808. Ten coded context neighbors are illustrated. Which values areincluded in the set of neighbors depends on the scan order. For example,using the zigzag forward scan order 702 of FIG. 7, the set of neighborsillustrated in FIG. 8 includes the coded context neighbors 804 whichincludes neighbors that are above and to the left of the current value802. For the current value 802, non_zero_map_sum(2,2)=5. This value(i.e., 5) can be used as context information to determine a probabilitymodel for coding the current value 802 of the non-zero map 800.

As indicated above, the coding context can also be based on the positionof the to-be-coded value within the non-zero map or, equivalently, inthe transform block. The positions of the transform block can be groupedinto context groups. For example, four context groups can be set: afirst group corresponding to the DC coefficient (i.e., r=0 and c=0), asecond group corresponding to the top row except for the AC coefficient(i.e., r=0 and c>0), a third group corresponding to the left-most columnexcept for the AC coefficient (i.e., r>0 and c=0), and a fourth groupcorresponding to all other coefficients (i.e., r>0 and c>0). As such,the current value 802 corresponds to the fourth context group.

In an implementation, encoding a non-zero map, at 602, can also includedetermining a coding context for each value of the end-of-block map. Theprocess 600 can determine a context model for a to-be-encoded value ofthe end-of-block map based on the location of the to-be-encoded valuewith respect to the frequency information of the transform block. Thatis, the position of the transform coefficient in the transform block canbe used as the context for determining the context model for encoding acorresponding (i.e., co-located) to-be-encoded value of the end-of-blockmap. The transform block can be partitioned into areas such that eacharea corresponds to a context. The partitioning can be based on therationale that the likelihood is very low that the end-of-block is atthe DC location of the transform block but that the likelihood increasesfurther from the DC coefficient.

In some implementations, a lower-range level map can be a binary maphaving dimensions corresponding to the dimensions of the transform blockand, as indicated above, a map level k. A position of the lower-rangelevel map can be set to one (1) when a corresponding value in thepreceding level map (i.e., level map k-1 as described below) is one (1)and the corresponding transform coefficient is greater than the maplevel k of the lower-range level map. A position of the lower-rangelevel map can be set to a value of zero when a corresponding value inthe preceding level map has a value of one and the correspondingtransform coefficient is equal to the map level k of the lower-rangelevel map. A position of the lower-range level map can have no valuewhen a corresponding value in the preceding level map has a value ofzero.

In an implementation of the process 600, encoding a lower-range levelmap for a level, at 604, can also include determining, based on a scanorder of the lower-range level map, a level-map coding context for avalue of the lower-range level map. As indicated above, encoding a valueof a lower-range level map k amount to encoding a binary value, namelywhether the corresponding (i.e., co-located) transform coefficient ofthe transform block is equal k or is above k. The encoding of binaryvalues results in simple contexts. As such, multiple neighboring valuesof a value can be used as the context for determining a context modelfor the value.

As also indicated above, scanning of the lower-range level map canproceed in a backwards scan order. As such, when encoding a value,neighboring values below and to the right of the to-be-encoded value(if, for example, the scan order is the zigzag forward scan order or 702of FIG. 7) will have already been encoded. Therefore, first neighboringvalues (e.g., below and right neighboring values) in the lower-rangelevel map can be used as context. Additionally, second neighboringvalues (e.g., top and left neighboring values) in the immediatelypreceding level-(k-1) map can also be used as context. The precedinglevel map of a lower-range level-k map is the lower-range level-(k-1)map, for k>2; and the preceding level map for the level-1 map is thenon-zero map.

As described above, the coding of the transform coefficients is amulti-pass process. In the first pass, the non-zero map 706, whichdescribes the locations of non-zero coefficients in the transform block,is coded following the forward scan order. In subsequent passes, thevalues of the non-zero coefficients following the backward scan order(i.e., from the position of the highest AC coefficient to the positionof the DC coefficient) are coded. Coding the non-zero map 706 can beimplemented using the steps:

-   -   1. Initialize i=0, where i denotes the scan position, and i=0        corresponds to the DC position (e.g., the transform coefficient        720).    -   2. Code a binary non-zero flag nz[i] indicating whether the        quantized transform coefficient at scan position i is zero. For        example, a zero value (nz[i]=0) can be coded when the quantized        transform coefficient is zero (i.e., the value in the non-zero        map 706 at scan position i is zero); otherwise (the value in the        non-zero map 706 at scan position i is 1), a one value (nz[i]=1)        is coded. In another example, a zero value (nz[i]=0) can be        coded when the quantized transform coefficient is not zero;        otherwise, a one value (nz[i]=1) is coded.    -   3. If nz[i] indicates that the transform coefficient at scan        position i is non-zero (e.g., nz[i]=1), then code a binary flag        indicating whether all the coefficients at scan positions higher        than i are all zero. That is, when a 1 value of the non-zero map        706 is coded, then a value at the same scan position in the        end-of-block map 726 is then coded.    -   4. Set i to the next scan position (i=i+1).    -   5. Repeat Steps 2-4 until EOB is met (i.e., until the        end-of-block value 730 is coded).    -   6. Set nz[j]=0 for all j>EOB. That is, set all transform        coefficients after the end-of-block value 730 to 0.

During the quantization process, such as described with respect to thequantization stage 406 of FIG. 4, a rate distortion optimizedquantization (RDOQ) process determines (e.g., calculates, selects,etc.), for transform coefficients of a transform block, respectivequantized transform coefficients according to a rate distortion cost ofeach of the quantized transform coefficients.

For example, in response to receiving a transform coefficient value x,the RDOQ may initially provide a quantized transform coefficient Q(x).The quantized transform coefficient Q(x) may be first obtained byminimizing the distortion (e.g., a loss in video quality). However, whenthe RDOQ considers the rate (e.g., a number of bits) of coding thequantized transform coefficient Q(x) in addition to the distortion, theRDOQ may obtain another quantized transform coefficient Q′(x) thatprovides a better overall rate distortion cost. This process cancontinue until an optimal quantized transform coefficient is obtainedfor the transform coefficient value x. As such, the quantizedcoefficient value of a transform coefficient may change during thecoding process of the transform coefficient and/or the transform blockthat includes the transform coefficient.

As described above, coding the non-zero map 706 uses a forward scanorder and coding the subsequent level maps uses a backward scan. Assuch, estimating the rate cost of changing a transform coefficient valuecan be difficult since the first pass and the second (or a subsequent)pass of coding use different scan orders: one forward and one backward.

More specifically, in the first pass where the scan order is forward(from the DC coefficient to the highest AC coefficient), a change to thequantized coefficient value at scan position i can impact the rate costof coding coefficients at scan positions j that follow the scan positioni (i.e., j>i); and in the second pass, where the scan order is backward(from the highest AC coefficient to the DC coefficient), a change to thequantized coefficient at scan position i can impact the rate cost ofcoding coefficients at scan positions j′ that precede the scan positioni (i.e., j′<i).

As such, to estimate the cost of coding a coefficient at scan positioni, information from transform coefficients at scan positions j>i andtransform coefficients at scan positions j′<i are required therebycreating a bi-directional dependency. This bi-directional dependency maysignificantly complicate the RDOQ process.

To avoid the bi-directional dependency, implementations according tothis disclosure can, instead of interleaving EOB indications (i.e.,end-of-block values of the end-of-block map 726) after non-zero valuesof the non-zero map 706, first code the EOB symbol and proceed toprocess the non-zero map 706 in a backward scan order. As such, backwardscan orders can be used for all passes of the coding of the transformblock using level maps. By using backward scan orders in all passes,only information from transform coefficients at scan positions jfollowing the scan position of a current transform coefficient i (i.e.,j>i) are required for estimating the rate cost of coding the coefficientat scan position i. As such, complexity is reduced, which in turns leadsto more efficient implementation of the RDOQ. Accordingly, coding atransform block using level maps can be implemented using the steps:

-   -   1. Code EOB.    -   2. Set nz[j]=0 for all j>EOB, and set nz[EOB]=1. Terminate the        process if EOB<1.    -   3. Initialize i=EOB−1.    -   4. Code nz[i] indicating whether the quantized transform        coefficient at scan position i is zero (nz[i]=0) or not        (nz[i]=1).    -   5. Set i=i−1.    -   6. Repeat Steps 3-5 until i=−1.

In the above steps, the EOB is as described with respect to FIGS. 6-7.That is the EOB indicates the location of the last non-zero coefficientof the transform block. However, other semantics for the EOB arepossible. For example, in an implementation, the EOB can indicate thelocation immediately after the last non-zero coefficient of thetransform block. As such, and referring to FIG. 7 for illustration, theEOB would indicate the scan position 12 (instead of the scan position 11as described with respect to FIGS. 6-7).

When the EOB indicates the position immediately after the last non-zerocoefficient, then the steps above can be given by the following:

-   -   1. Code EOB.    -   2. Set nz[j]=0 for all j≥EOB, and set nz[EOB−1]=1. Terminate the        process if EOB≤1.    -   3. Initialize i=EOB−2.    -   4. Code nz[i] indicating whether the quantized transform        coefficient at scan position i is zero (nz[i]=0) or not        (nz[i]=1).    -   5. Set i=i−1.    -   6. Repeat Steps 3-5 until i=1.

FIG. 9 is a flowchart diagram of a process 900 for coding a transformblock using level maps according to an implementation of thisdisclosure. The process 900 can be implemented by an encoder such as theencoder 400 of FIG. 4. When implemented by an encoder, coding meansencoding in an encoded bitstream, such as the compressed bitstream 420of FIG. 4. For example, the process 900 can be performed in whole or inpart by the entropy encoding stage 408 of the encoder 400. The process900 can be performed by a decoder such as the decoder 500 of FIG. 5.When implemented by a decoder, coding means decoding from an encodedbitstream, such as the compressed bitstream 420 of FIG. 5. For example,the process 900 can be performed in whole or in part by the entropydecoding stage 502 of the decoder 500 and the encoded video bitstreamcan be the compressed bitstream 420 of FIG. 5.

Implementations of the process 900 can be performed by storinginstructions in a memory such as the memory 204 of the receiving station106 to be executed by a processor such as CPU 202, for example.

The process 900 can be implemented using specialized hardware orfirmware. Some computing devices can have multiple memories, multipleprocessors, or both. The steps or operations of the process 900 can bedistributed using different processors, memories, or both. Forsimplicity of explanation, the process 900 is depicted and described asa series of steps or operations. However, the teachings in accordancewith this disclosure can occur in various orders and/or concurrently.Additionally, steps in accordance with this disclosure may occur withother steps not presented and described herein. Furthermore, not allillustrated steps or operations may be used to implement a method inaccordance with the disclosed subject matter.

At 902, the process 900 codes an end-of-block indicator of the transformblock. In an implementation, the scan position of the EOB can be coded.For example, and referring to FIG. 7, the scan position 11 correspondingto the last non-zero transform coefficient 708 can be coded. In anotherexample, the scan position 12, corresponding to the coefficientfollowing the last non-zero transform coefficient 708 in the forwardscan order, is coded. In an implementation, the end-of-block indicatorcan be coded using a context model.

At 904, the process 900 codes the non-zero map in the backward scanorder starting at the last non-zero coefficient of the transform block.The non-zero map indicates which transform coefficients of a transformblock have a zero value and which transform coefficients of thetransform block have a non-zero value. The process 900 codes thenon-zero map that is similar to the non-zero map 706 of FIG. 7. Theprocess 900 codes a binary value indicating whether a quantizedtransform coefficient at a scan position of the scan order is zero ornon-zero. For example, for a scan position i, the process 900 can code azero if the quantized transform coefficient at the scan position i iszero; otherwise a 1 is coded.

At 906, the process 900 codes a respective lower-range level map havinga respective map level up to a maximum level T. A lower-range level maphaving to a map level indicates the transform coefficients of thetransform block are equal, in absolute value, to the respective maplevel and which transform coefficients of the transform block are, inabsolute value, greater than the respective map level. When implementedby a decoder, the process 900 decodes values from the encoded videobitstream to reconstruct lower-range level-k maps encoded as describedwith respect to 604 of the process 600.

For example, to reconstruct a level-1 map, the process 900 starts fromthe highest non-zero transform coefficient traversing backwards todetermine which of transform coefficients are equal to 1 and which aregreater than 1. That is, using the reconstructed non-zero map of thenon-zero map 706 of FIG. 7, and starting at the last non-zerocoefficient 710 and traversing backwards to the value 740, the process900 reconstructs the level-1 map 707 of FIG. 7. For each 1 value of thereconstructed non-zero map, the process 900 decodes a value from theencoded video bitstream and reconstructs the level-1 map 707. The valuesdecoded by the process 900 are zero and one (1) values.

To reconstruct a level-2 map, the process 900 uses the same procedure asthat used to generate the level-1 map except that, instead of traversingthe reconstructed non-zero map, the process 900 uses the reconstructedlevel-1 map. The process 900 repeats all the steps until the maximumlevel number of level maps are reconstructed.

In an implementation, the maximum level T can be provided to the process900 via a configuration. In another implementation, the maximum level Tcan be signaled, by an encoder, in the encoded video bitstream. As such,the process 900 decodes the maximum level T from the encoded videobitstream.

At 908, the process 900 codes a coefficient residual map. Each residualcoefficient of the coefficient residual map corresponds to a respectivetransform coefficient of the transform block having an absolute valuethat exceeds the maximum level. When implemented by a decoder, theprocess 900 reconstructs, e.g., the coefficient residual map 734 of FIG.7. For each one (1) value of the level-T map, the process 900 decodes acorresponding residual value from the encoded bitstream to reconstructthe coefficient residual map 734 of FIG. 7.

In some implementations, the process 900 can include coding a sign map,at 910. The sign map can be a sign map such as described with respect tothe sign map 732 of FIG. 7. The sign map indicates which transformcoefficients of the transform block have positive values and whichtransform coefficients have negative values.

For a transform block of size N×N, in the worst case N×N−1 bins (binarysymbols) may need to be context coded to determine the EOB position. Forexample, when N=32, in the worst cost, a total of 1023 bins may becontext coded (i.e., coded using a context model) to determine the EOBposition.

Some implementations can use scan positions groups to reduce the numberof context-coded bins required to code the EOB position. As such, codingthe scan position corresponding to the end-of-block position can includecoding an index of a scan positions group that includes the scanposition and coding an offset within the scan positions group, theoffset corresponding to a position of the scan position within the scanpositions group.

In an implementation of level maps, the value EOB=0 can be reserved forindicate that all transform coefficients of the block are zero. That is,when EOB=0, then the block is an all-zero block.

In an example of such implementations, the scan positions can bepartitioned (e.g., grouped) into 11 scan positions groups: 1, 2, [3, 4],[5-8], [9-16], [17-32], [33-64], [65-128], [129-256], [257-512],[513-1024]. That is, the group with index 0 includes only the scanposition 1; the group with index 1 includes only the scan position 2;the group with index 4 includes the scan positions 9-16; and so on. Forexample, assuming that the scan position 50, corresponding to the EOB,is to be coded, then the index 6 of a scan positions group [33-64] thatincludes the scan position 50 is coded and the offset 17 (i.e.,50−33=17) within the scan positions group [33-64] is coded.

In an implementation, the index of the group that includes the codedscan position corresponding to the end-of-block is context-coded (i.e.,is coded using arithmetic coding using a context model) and the offsetwithin the scan positions group can be coded in by-pass mode. By-passmode, which may also be referred to as the literal mode, means that thevalue to be coded is not coded using a context model. The by-pass modecan be used, for example, when the offset values within a range areequally probable. To code the offset 17, five (5) bits are required. Assuch, in this example, the number of context coded bins for EOB is atmost 10 corresponding to the group indexes {0, 1, . . . , 10}.

In an implementation, the index of the group that includes the codedscan position corresponding to the end-of-block is context-coded (i.e.,coded using a context model) and at least some of the most significantbits of the offset within the scan positions group can also becontext-coded. That is, the offset value can be considered to includeprefix bits (i.e. most significant bits) and suffix bits (i.e., leastsignificant bits). The prefix bits can be context-coded and the suffixbits can be coded using by-pass mode. Any number of bits of the offsetvalue can be considered the most significant bits. For example, theoffset value 17 corresponds to the binary string 10001. If the first 2bits are considered most significant, then the bits 10 are context-codedand the bits 001 are by-pass coded (i.e., coded using by-pass mode). Ifthe first 3 bits are considered most significant, then the bits 100 arecontext-coded and the bits 01 are by-pass coded.

The scan positions can be grouped into scan position groups in anynumber of ways. In an implementation, and as illustrated by the abovegroup, each group can include a power of 2 number of scan positions. Thepower of 2 can be the index scan position groups minus 1 (i.e., index−1). For example, the scan position groups at index 5 (i.e., the group[17-32]) includes 2⁵⁻¹ (=2⁴=16) scan positions. As such, to code anoffset within a scan position groups having an index inx, only (inx−1)bits are required. For example, one (1) bit is required to code anoffset in the group having index 2 (the group [3,4]), two (2) bits arerequired to code an offset in the group having index 3 (the group[9-16]), and so on.

In an implementation, the number of scan positions in each scanpositions group can be limited to a maximum predetermined number (i.e.,a ceiling). For example, the group size can be limited to no greaterthan 16. As such, the above group can be modified as follows: 1, 2, [3,4], [5-8], [9-16], [17-32], [33-48], [49-64], . . . , [1009-1024]. Assuch, coding an offset requires no more than 4 bits. To code an EOBposition of 50 using the modified groups, the index 7, which correspondsto the scan positions group [49-64], can be coded using arithmeticcoding with context models. The offset 1 (=50−49) can be coded in bypassmode by using 4 bits, namely 0001.

In some implementations, the value EOB=0 is not reserved for indicatingthat all transform coefficients of the block are zero. In suchimplementations, the scan positions groups can start at 0 and end at1023, such as 0, 1, [2, 3], [4-7], [8-15], [16-31], [32-63], [64-127],[128-255], [256-511], [512-1023].

As described above, the values of the non-zero map 706 include binaryvalues. The binary values indicate whether a transform coefficient at agiven location of the transform block is zero or non-zero. As such, thevalues of the non-zero map can be considered non-zero flags. The binaryvalues enable the use of sophisticated spatial neighboring templates forcontext modeling. Such spatial neighboring templates can better capturestatistical characteristics of transform blocks, especially those withlarger transform block sizes. Accordingly, the coding of non-zero flags(and thus, the coding of transform coefficients) can be improved byfully utilizing the information of neighboring non-zero flags whendetermining a context for selecting a context model.

A template captures the coded history of the non-zero flags (i.e.,values of the non-zero map) that are coded before a current non-zeroflag. A template can define (e.g., specify, select, set, or any waydefine.), for a current scan position, the scan positions of thenon-zero map values that are to be used for determining the context forcoding the current value. Equivalently, a template can be defined interms of the Cartesian coordinates, within the non-zero map, of thenon-zero values to be used for determining the context.

FIGS. 10A-10B is a diagram of examples 1000 of templates for determininga coding context according to implementations of this disclosure. InFIGS. 10A-10B, values (i.e., circles representing values of a non-zeromap) shaded using the pattern 1004 are to-be-coded values; and thevalues shaded with the pattern 1002 are values for which contextinformation are available because these values are coded before ato-be-coded value 1032. In the examples of FIGS. 10A-10B, theto-be-coded value 1032 depicts the current value of the non-zero map tobe coded. The examples of FIGS. 10A-10B are non-limiting examples.Templates having other shapes and/or sizes are also possible.

In an example, the number of non-zero values corresponding to thetemplate positions can be used as the context for coding the currentvalue. For example, the values corresponding to the template positionscan be added and the sum can be used as the context. In some cases, atemplate position may not be available, such as, for example, if thecontext position is outside the boundaries of the block. In an example,an unavailable value can be assumed to be zero (0). In another example,an unavailable value can be assumed to be one (1).

A codec can use multiple transform types. For example, a transform typecan be the transform type used by the transform stage 404 of FIG. 4 togenerate the transform block. For example, the transform type (i.e., aninverse transform type) can be the transform type to be used by thedequantization stage 504 of FIG. 5. Available transform types caninclude a one-dimensional Discrete Cosine Transform (1D DCT) or itsapproximation, one-dimensional Discrete Sine Transform DST (1D DST) orits approximation, a two-dimensional DCT (2D DCT) or its approximation,two-dimensional DST (2D DST) or its approximation, and an identitytransform. Other transform types can be available. In an example, aone-dimensional transform (1D DCT or 1D DST) can be applied in onedimension (e.g., row or column) and the identity transform applied inthe other dimension.

In the cases where a 1D transform (e.g., 1D DCT, 1D DST) is used (e.g.,1D DCT is applied to columns (or rows, respectively) of a transformblock), the quantized coefficients can be coded by using a row-by-row(i.e., raster) scanning order or a column-by-column scanning order. Inthe cases where 2D transforms (e.g., 2D DCT) are used, a differentscanning order may be used to code the quantized coefficients. Asindicated above, different templates can be used to derive contexts forcoding the non-zero flags of the non-zero map based on the types oftransforms used. As such, in an implementation, the template can beselected based on the transform type used to generate the transformblock. As indicated above, examples of a transform type include: 1D DCTapplied to rows (or columns) and an identity transform applied tocolumns (or rows); 1D DST applied to rows (or columns) and an identitytransform applied to columns (or rows); 1D DCT applied to rows (orcolumns) and 1D DST applied to columns (or rows); a 2D DCT; and a 2DDST. Other combinations of transforms can comprise a transform type.

As indicated above with respect to FIG. 9, the non-zero map can be codedin the backward scan order starting at the last non-zero coefficient(i.e., starting at the highest AC transform coefficient) of thetransform block. As such, the coded history of a current value (i.e., acurrent non-zero flag) of the non-zero map includes values that are tothe right and below the current value in the two-dimensional non-zeromap, such as the non-zero map 706.

When a 1D vertical transform type is applied, the to-be-coded value 1032is more correlated with vertical neighbor values than with horizontalneighbor values. As such, a template 1010 can be used in the case a 1Dtransform (e.g., 1D DCT) is applied to columns. Assuming that theto-be-coded value 1032 is at position (x, y) of the non-zero map, thenthe template 1010 comprises the values at the seven positions (x+1, y),(x+2, y), (x+3, y), (x+4, y), (x, y+1), (x+1, y+1), and (x+1, y+2).

When a 1D horizontal transform type is applied, the to-be-coded value1032 is more correlated with horizontal neighbor values than withvertical neighbor values. As such, a template 1020 can be used in thecase a 1D transform (e.g., 1D DCT) is applied to rows. Assuming that theto-be-coded value 1032 is at position (x, y) of the non-zero map, thenthe template 1020 comprises the values at the seven positions (x+1, y),(x, y+1), (x+1, y+1), (x, y+2), (x+1, y+2), (x, y+3), and (x, y+4).

When a 2D transform type (e.g., 2D DCT, 2D DST) is applied, a template1030 can be used. Assuming that the to-be-coded value 1032 is atposition (x, y) of the non-zero map, then the template 1030 comprisesthe values at the seven positions (x+1, y), (x+2, y), (x, y+1), (x+1,y+1), (x+2, y+1), (x, y+2), and (x+1, y+2).

In some examples of templates, the non-zero value that is scannedimmediately before the to-be-coded value 1032 is not included in thetemplate. That is, if the to-be-coded value 1032 is at scan position i,then the non-zero value at scan position (i−1) is not included in thetemplate. Even though the non-zero map is coded in a backward scanorder, the scan order of the scan order can depend on the transformtype. The scan order of the backward scan order is the order in whichnon-zero values of the non-zero map are visited from the highest ACvalue to the DC value. In an example, a vertical scan order can be usedwhen a 1D horizontal transform type is used. As such, the scan orderproceeds in column-wise order (e.g., from bottom to top). In an example,a horizontal scan order can be used when a 1D vertical transform type isused. As such, the scan order proceeds in row-wise order (e.g., fromright to left).

Template 1040 is another example of a template that can be used when a1D-transform type is applied to columns. In the template 1040, thenon-zero value that is scanned immediately before the to-be-coded value1032 is not included in the template. Assuming that the to-be-codedvalue 1032 is at position (x, y) of the non-zero map, then the template1040 comprises the values at the seven positions (x+1, y), (x+2, y),(x+3, y), (x+4, y), (x+1, y+1), (x+1, y+1), and (x+1, y+2).

Template 1050 is another example of a template that can be used when a1D-transform type is applied to rows. In the template 1050, the non-zerovalue that is scanned immediately before the to-be-coded value 1032 isnot included in the template. Assuming that the to-be-coded value 1032is at position (x, y) of the non-zero map, then the template 1040comprises the values at the seven positions (x, y+1), (x+1, y+1), (x,y+2), (x+1, y+2), (x, y+3), (x+1, y+3), and (x, y+4).

Template 1060 is an example of a template that can be used when a 2Dtransform (e.g., 2D DCT, 2D DST) is used. Assuming that the to-be-codedvalue 1032 is at position (x, y) of the non-zero map, then the template1040 comprises the values at the seven positions (x+1, y), (x+2, y),(x+3, y), (x, y+1), (x+1, y+1), (x, y+2), and (x, y+3).

Each of templates 1010-1060 includes seven (7) positions. However, atemplate can include more or less positions and/or can have othershapes. For example, template 1070 is another example of a template thatcan be used when a 2D transform type is used. The template 1070 includeseight (8) positions. Assuming that the to-be-coded value 1032 is atposition (x, y) of the non-zero map, then the template 1040 comprisesthe values at the eight positions (x+1, y), (x+2, y), (x+3, y), (x,y+1), (x+1, y+1), (x, y+2), (x+1, y+2), and (x, y+3).

In other examples, a template can include five (5) positions. Forexample, a template 1088 of FIG. 10B, which includes the positions (x+1,y), (x, y+1), (x+2, y), (x, y+2), (x+1, y+1), can be used for2D-transform types. For example, templates 1080, 1082, 1083 can be usedfor a vertical 1D-transform type. The template 1080 includes thepositions (x+1, y), (x+2, y), (x+3, y), (x+4, y), and (x, y+1). Thetemplate 1082 includes the positions (x+1, y), (x+2, y), (x+3, y), (x+4,y), and (x+1, y+1). The template 1083 includes the positions (x, y+1),(x, y+2), (x, y+3), (x+1, y), (x+1, y+1). For example, templates 1084,1086, and 1087 can be used for a horizontal 1D-transform type. Thetemplate 1084 includes the positions (x+1, y), (x+2, y), (x+3, y), (x,y+1), (x+1, y+1). The template 1086 includes the positions (x, y+1), (x,y+2), (x, y+3), (x, y+4), and (x+1, y+1). The template 1087 includes thepositions (x, y+1), (x, y+2), (x, y+3), (x, y+4), and (x+1, y). In someimplementation, the positions (x+1, y) and (x, y+1) can be replaced bythe positions (x+1, y+2) and (x+2, y+1), respectively.

In some implementations, coding of a level-k map, where k>0, can alsouse different templates (e.g., templates as described above) dependingupon transform types. For example, as described above, one template maybe used for 2D-transform type, one template may be used for vertical1D-transform type, and another template may be used for horizontal1D-transform type.

In some implementations, the contexts used to code the non-zero flagscan depend upon the locations (i.e., the scan positions or the blockpositions) of the non-zero flags. As such, in some implementations, thetransform-type-dependent templates, described above, can be combinedwith the locations of the non-zero flags to determine a context. Toavoid using too many contexts, which may lead to the so-called contextdilution problem, the locations can be classified into regions. Forexample, the classification may be dependent upon the transform type.

As described above with respect to FIG. 7, the level maps are codedsequentially. That is, the non-zero map 706 is coded, then the level-1map is coded, then the level-2 map is coded, and then the coefficientresidual map 734 is coded. However, in some implementations, a differentcoding structure can be used.

As described above with respect to FIGS. 10A-10B, coding of a level-kmap, where k>0, can also use different templates (e.g., templates asdescribed above) depending upon transform types. That is, a template, asdescribed above can be used to determine a context for coding whether acoefficient at (x, y) is greater than 1 (e.g., using a correspondingvalue of a level-1 map, such as the level-1 map 707 of FIG. 7) or isgreater than 2 (e.g., using a corresponding value of a level-2 map, suchas the level-2 map 709 of FIG. 7).

As such, each coefficient can be coded up to whether the coefficient isgreater than the maximum level. In the example of FIG. 7, the maximumlevel is 2. As such, each coefficient can be coded up to whether thecoefficient is greater than 2 (using corresponding values of level maps)before proceeding to coding of a next coefficient. In an implementation,a coefficient value that is greater than 2 (i.e., a coefficient valuethat is greater than the maximum level) can be represented by the value3 (i.e., the maximum level +1). As such, coding a coefficient “up towhether the coefficient is greater than maximum level (e.g., 2)” (i.e.,coding an “up-to” value) can mean coding a value 0, 1, 2, . . . ,(maximum level+1) (e.g. 3) corresponding, respectively and in the casewhat the maximum level is 2, to a coefficient having a value equal to 0,equal to 1, equal to 2, and greater than 2. For brevity, and asmentioned above, a value that results from coding a coefficient “up towhether the coefficient is greater than maximum level” is referred toherein as an “up-to” value.

The “up-to” values of the coefficients of the transform block can bereferred to collectively as a “lower plane” or a first plane. Thecoefficient residual map, such as the coefficient residual map 734 ofFIG. 7, is also referred to herein as the “higher plane” or secondplane.

FIG. 11 is a flowchart diagram of a process 1100 for coding a transformblock using level maps according to an implementation of thisdisclosure. Unlike the process 900 which codes the level mapssequentially (i.e., each map is coded before proceeding to coding thenext map), the process 1100 codes, for each non-zero coefficient, usinga template, and in a case where the maximum level is 2 (i.e., T=2),whether the coefficient is 0, 1, 2, or greater than 2 (represented bythe value 3). That is, the process 1100 codes the “up-to” value of thelower plane of a coefficient before proceeding to the next coefficientin the scan order. The process 1100 can include blocks similar to thoseof the process 900. Descriptions of the similar blocks (e.g., 902, 908,and 910) are omitted. Some implementations of the process 1100 caninclude the block 910 before the block 908.

The process 1100 can be implemented by an encoder such as the encoder400 of FIG. 4. When implemented by an encoder, coding means encoding inan encoded bitstream, such as the compressed bitstream 420 of FIG. 4.For example, the process 1100 can be performed in whole or in part bythe entropy encoding stage 408 of the encoder 400. The process 1100 canbe performed by a decoder such as the decoder 500 of FIG. 5. Whenimplemented by a decoder, coding means decoding from an encodedbitstream, such as the compressed bitstream 420 of FIG. 5. For example,the process 1100 can be performed in whole or in part by the entropydecoding stage 502 of the decoder 500 and the encoded video bitstreamcan be the compressed bitstream 420 of FIG. 5.

Implementations of the process 1100 can be performed by storinginstructions in a memory such as the memory 204 of the receiving station106 to be executed by a processor such as CPU 202, for example.

The process 1100 can be implemented using specialized hardware orfirmware. Some computing devices can have multiple memories, multipleprocessors, or both. The steps or operations of the process 1100 can bedistributed using different processors, memories, or both. Forsimplicity of explanation, the process 1100 is depicted and described asa series of steps or operations. However, the teachings in accordancewith this disclosure can occur in various orders and/or concurrently.Additionally, steps in accordance with this disclosure may occur withother steps not presented and described herein. Furthermore, not allillustrated steps or operations may be used to implement a method inaccordance with the disclosed subject matter.

At 1102, the process 1100 determines whether there are more non-zerocoefficients to code. If so, the process 1100 proceeds to 1104 to code acurrent quantized transform coefficient at (x, y); otherwise, theprocess 1100 proceeds to 908 followed by 910. At 1104, the process 1100selects a template for coding a current quantized transform coefficient.As used in this disclosure, “select” means to identify, construct,determine, specify, generate, or other select in any manner whatsoever.The template can be a template as described with respect to FIGS.10A-10B. In an example, the same template is used for coding all thecoefficients of the transform block. As such, the template can beselected once for the process 1100 and be performed before the block1102.

At 1106, the process 1100 determines a context using the template. Theprocess 1100 can determine the context using the template in any numberof ways. Each template position corresponds to a lower plane value(e.g., 0, 1, . . . , T+1). Combinations of the values can be used todetermine the context. For example, a sum of the values can be used. Forexample, a weighted sum can be used to determine the context. The weightassigned to a position can be set based on the distance to an “origin.”In an example, the origin can be the location (x, y) of the currenttransform coefficient for which the context is being determined.Examples of distance include a scan position distance (e.g., adifference between the scan position of the current coefficient and aposition of the template) or a Cartesian distance. However, other waysof setting the weight can be available. In yet another example, anon-linear function can be used. For examples, the maximum or minimumvalue in the template can be used for determining the context. In yetanother example, the context can be determined using a combination ofthe sum and the maximum values. Other methods and/or values, orcombinations of methods and/or values, can be used for determining thecontext from the template.

Using the sum (i.e., addition) of the values at the positions of thetemplate to determine a context is now given. It should be understoodthat the following can be used with any method for determining thecontext using the template.

The process 1100 can add up (sums) the values corresponding to thepositions of the template. When using a template for deriving a contextfor coding a coefficient, each of the positions of the template can haveone of the values 0, 1, 2, or 3 (i.e., when T=2). As such, if thetemplate includes N positions, then the maximum sum can be 3*N. Forexample, if a template that includes 5 positions is selected at 1104,then the maximum sum can be 15 (=3*5); if a template that includes 7positions is selected, then the maximum sum can be 21 (=3*7).

In an implementation, where level maps as described with respect to FIG.7 are used, to determine a value for a position of the template, thevalues, at the same positions in the non-zero maps and the level-k mapscan be added or counted. For example, assuming that the position of thelast non-zero transform coefficient 708 is a position of the template,then the value of the template at that position can be determined to bethe sum of the values at 710 (i.e., 1) and 716 (i.e., 0) of the non-zeromap 706 and level-1 map 707. As such, the value is 1. As another,assuming that the position of the transform coefficient 739 is aposition of the template, then the value of the template at thatposition can be determined to be the sum of the values at thecorresponding locations of in the non-zero map 706, the level-1 map 707,and the level-2 map 709. As such, the value is 3. As such, the contextindex can be determined using a sum of the values corresponding topositions of the template where each value of the template is determinedby summing respective values of at least some of the level maps.

In another implementation, the level maps, including the zero map and asdescribed with respect to FIG. 7 are not generated. Instead, a lowerplane one-dimensional array, level, can be used to indicate whether atransform coefficient is one of 0, 1, . . . , T+1. Whereas the lowerplane mentioned above is a 2-dimensional structure corresponding to thetransform block, the lower plane one-dimensional array, level, isgenerated as a result of the scan order. The lower plane one-dimensionalarray, level, is referred to as the lower-plane array.

Given a scan position i, level[i]={0, 1, . . . , T+1}. The lower planearray, level, can include values for all transform coefficients of thetransform block. Alternatively, the lower plane array can include valuesfor coefficients up to the end-of-block coefficient. That is, the lowerplane array, level, can include values for each transform coefficient upto and including the last non-zero coefficient of the transform block.

In the case of an encoder, the lower plane array, level, can begenerated a priori before coding its values in a bitstream.Alternatively, the lower plane array, level, can be generated ascoefficients are encoded. For example, the lower plane array can begenerated using the steps: 1) for a scan position, determine acorresponding “up-to” value, 2) code the “up-to” value, and 3)store/place the “up-to” at the scan position in the lower plane, level.The steps 2 and 3 can be reversed. In the case of a decoder, the lowerplane array, level, can be populated with values as the values aredecoded from the bitstream.

The sum for determining the context index can be calculated by addingthe respective values of the lower plane array, level. For example,assuming that the template includes 5 positions corresponding to thescan positions l₁, l₂, l₃, l₄, and l₅, then the sum can be determined assum=level[l1] +level[l2]+level[l3]+level[l4]+level[l5].

Using the sum, the process 1100 can determine a context index (e.g., avariable ctx). The context index can be determined using an operationthat is similar to the operation ((counts+1)>>1) as described above.However, instead of using “count,” a “sum” is used. As such, the process1100 uses ((sum+1)>>1) for determining the context index.

The context index ctx may be out of range. For example, assuming thatthe sum is 15, the transform class is TX_CLASS_2D, and that (x, y) is(1, 2), then the context index is ((sum+1)>>1)+6=((15+1)>>1)+6=14.However, the number of available contexts for the TX_CLASS_2D, using theabove example, is 11. As such, the context index is out of range.Equivalently, a sum that results in a context index that is out of rangecan itself be considered out of range. If the context index is out ofthe range, then, the process 1100 can set the context index to apredetermined number. As such, the process 1100 can determine thecontext index using a formula such as min(((sum+1)>>1), predeterminednumber). As such, the value ((sum+1)>>1) is upper-bounded by thepredetermined number. In an example, the predetermined number can be 4.In an example, the predetermined number can depend on the transformclass type. The predetermined number can be selected in any other ways.

The context index ctx can be used to select a context for coding thecurrent transform coefficient; more specifically, to code the “up-to”value of the current transform coefficient. At 1108, the process 1100codes the coefficient using the context.

An implementation of the process 1100 that uses the lower plane array,level, for coding transform coefficients of the transform block can besummarized using the following procedure:

 1. for (i=eob−1; i>=0; i−−) {  2.  if (i < eob−1) code level[i]>0;  3. if (level [i] > 0) {  4.   for (j=0; j<T; j++) {  5.    code level[i] >j+1  6.    if (level[i] == j+1)  7.     break;  8.   }  9.  } 10. } 11.for (i==0; i < eob; i++ { 12.  if level[i] != 0 13.   code sign[i] 14. }15. for (i=eob−1; i >= 0; i−−) 16.  if level[i] > T 17.   codelevel[i]−T−1

The steps 1-10 can be repeated for each transform coefficient up to thelast non-zero coefficient of the transform block. For a transformcoefficient at scan position i, the steps 2-9 code the “up-to” value.That is, the steps 2-9 code “up to whether a transform coefficient ofthe transform block is greater than a maximum level of the level maps.”For example, assuming that the maximum level T=2 and i<eob-1, iflevel[i]=3, then the steps 2-9 code the bits 111; if level[i]=0, thenthe steps 2-9 code the bit 0; and if level[i]=2, then the steps 2-9 codethe bits 110. The steps 11-14 code the sign bits (e.g., the values ofthe sign map 732 of FIG. 7) of the non-zero transform coefficients up tothe last non-zero coefficient. The steps 15-17 code, for each non-zerocoefficient that is greater than the maximum level T (i.e., level[i]>T),a residual for the transform coefficient. That is, the steps 15-17 codea value from the higher plane. The values of the residuals can be asdescribed with respect to the coefficient residual map 734.

Blocks 1102-1108 of FIG. 11 correspond to coding the lower plane andblock 908 corresponds to coding the higher plane described above.

Using the coding structure described in FIG. 11 for coding a transformcoefficient (i.e., a quantized transform coefficient), as compared tothe coding structure described with respect to FIG. 9, better throughputcan be obtained. Better throughput can mean that a transform block canbe decoded faster using the coding structure of the process 1100 thanthat of the process 900.

FIG. 12 is an example of an illustration 1200 of using a template dodetermine a context according to implementations of this disclosure. Theillustration 1200 uses a template, as described with respect to FIGS.10A-10B, to determine a context index for a to-be-coded value 1032. Thetemplate used in the illustration 1200 is the template 1088 of FIG. 10B.However, other templates, such as the template 1070, can be used. Theto-be-coded value can be the to-be-coded value 1032 of FIGS. 10A-10B. InFIG. 10B, the neighbors of the to-be-coded value 1032 are shown ashaving actual values. In FIG. 12, the neighbors of the to-be-coded value1032 are numbers (i.e., 1-5) for ease of reference.

As described with respect to FIG. 11, coding proceeds in a backward scanorder and the lower plane array, level, can be created. In the case ofan encoder, “coding” means encoding, and a “to-be-coded” value means avalue that is to be added to a compressed bitstream, such as thecompressed bitstream 420 of FIG. 4. In the case of a decoder, “coding”means decoding, and a “to-be-coded” value means a value that is to beread from, or decoded from, a compressed bitstream, such as thecompressed bitstream 420 of FIG. 5. The values of the lower plane array,level, are arranged according to the scan order.

The scan order determines a mapping from scan order positions (i.e.,locations) of the one-dimensional lower plane array, level, to Cartesianlocations of the transform block, or the lower plane; equivalently, thescan order determines a mapping from the Cartesian locations to scanorder positions.

As described above, the template indicates, for a to-be-coded value ofthe lower plane, the scan positions of coded levels of the lower plane.Equivalently, as there is a one-to-one correspondence between scanpositions and Cartesian coordinates for a given scan order, the templatecan indicate Cartesian coordinates. The coordinates can be absolutecoordinates, using the DC coefficient as the origin. Alternatively, thecoordinates can be relative coordinates (e.g., defined as vertical andhorizontal offsets from the to-be-coded value). The neighbors specifiedby the template are arranged along template anti-diagonal lines. Forexample, and referring again to the template 1088 of FIG. 12, neighbors1 and 3 are arranged along a template anti-diagonal line 1202; and theneighbors 2, 4, and 5 are arranged along a template anti-diagonal line1204. As such, and depending on the number of neighboring locations ofthe template, the scan positions indicated by the template are arranged,in the template, in at least two template anti-diagonal lines. Forexample, while the scan positions indicated by the template 1088 arearranged along two template anti-diagonal lines, the scan positionsindicated by the template 1070 of FIG. 10A are arranged along threetemplate anti-diagonal lines.

An illustration of using the template 1088 with different scan orders isnow given. The illustration uses a 4×4 block; however, the disclosure isnot limited to any block size. Using a backward vertical scan order 1208to process a 4×4 lower plane 1216, the value of the one-dimensionallower plane at a scan position 1210 (i.e., the scan position 14 in thescan order) corresponds to the lower plane value that corresponds to thetransform coefficient at Cartesian location (1, 0) of the transformblock. Assuming that the scan position 1210 is the to-be-coded value1032, and superimposing the template 1088 on the backward vertical scanorder 1208, then the neighboring locations 1-5 of the template 1088correspond, respectively, to the scan positions 10, 6, 13, 9, and 12.Sorted in ascending order, the neighboring scan positions are 6, 9, 10,12, and 13. As can be seen, the scan positions are not contiguous. Thatis, the scan positions are not sequential.

As another example, using the backward horizontal scan order 1212, thesame Cartesian location (1, 0) corresponds to a scan position 11 (thescan position 11 in the scan order). Assuming that the scan position1214 is the to-be-coded value 1032, and superimposing the template 1088on the backward horizontal scan order 1212, then the neighboringlocations 1-5 of the template 1088 correspond, respectively, to the scanpositions 10, 9, 7, 6, and 3. Sorted in ascending order, the neighboringscan positions are 3, 6, 7, 9, and 10. Again, the scan positions of theneighboring locations are not contiguous (i.e., sequential).

Lower plane array 1206 illustrates a hypothetical ordering of values ofa lower plane (not shown) according to a hypothetical scan order. As canbe seen in the lower plane array 1206, the example of the backwardvertical scan order 1208, and the example of the backward horizontalscan order 1212, the neighboring values 1-5 of the to-be-coded value1032 using the template 1088 are located sparsely (i.e.,non-contiguously) in the 1-dimensional lower plane array, level. Assuch, deriving a context model using the five neighbors of the template1088 requires accessing sparse points in the 1-dimensional array.

It is noted that for nearly all computing platforms, including x86, ARM,and hardware-implemented codecs, the 2-dimensional array transformcoefficient is stored in 1-D array (also referred to as line buffer).Lower plane arrays can also stored in line buffers. As accessingnon-continuous (i.e., non-contiguous) data from line buffers downgradesdata caching performance, using scan orders and/or techniques wherebyneighboring values are located contiguously in line buffers can improvecoding performance.

FIG. 13 is a diagram of an example 1300 of a scan order that alignsvalues along anti-diagonal lines according the implementations of thisdisclosure. The example 1300 includes a block 1302. The block 1302 canbe a transform block from which lower plane values can be determined.That is, a lower plane block is not generated, rather the values of thelower plane block are inferred from the transform coefficients.Alternative, the block 1302 can represent a lower plane. That is, thelower plane can be generated and stored in memory from the transformcoefficients. For simplicity of explanation, the block 1302 is assumedto be the 2-dimensional lower plane.

The block 1302 is shown as an 8×8 block. However, this disclosure is notlimited to any particular block size. As indicated above, the lowerplane would include, at a Cartesian location, a value for a transformcoefficient at the same Cartesian location of the transform block.However, for illustration purposes, the cells of the block 1302 show thescan position of the respective cell, instead of the particular value atthat cell. As indicated above, a lower plane, such as the block 1302 canbe generated as values an encoded in a bitstream. Accordingly, the block1302 may not be fully populated with values. For example, if a currentscan position is x, then the block 1302 may only include values for scanpositions lower than (or lower than and including) scan position x.

The scan order of the example 1300 can be referred to as ananti-diagonally aligned scan order. Using the anti-diagonally alignedscan order, a codec can advantageously process (to determine contextindexes) all values in one scan-order anti-diagonal line using efficientparallel computing thereby improving cache performance.

Four scan-order anti-diagonal lines are indicated in the example 1300. Ascan-order anti-diagonal line 1304 includes the scan positions 54-57; ascan-order anti-diagonal line 1306 includes the scan positions 49-53; ascan-order anti-diagonal line 1308 includes the scan positions 43-48;and a scan-order anti-diagonal line 1310 includes the scan positions36-42.

Each scan-order anti-diagonal line is a line such that levels in theblock 1302 having the same value col+row are considered to be on thesame scan-order anti-diagonal line. For example, the scan-orderanti-diagonal line 1304 includes those values having col+row=3. As such,the scan-order anti-diagonal line 1304 includes the values correspondingto the scan position 54 (corresponding to the value at block 1302location (0, 3)), the scan position 55 (corresponding to the value atblock 1302 location (1, 2)), the scan position 56 (corresponding to thevalue at block 1302 location (2, 1)), and the scan position 57(corresponding to the value at block 1302 location (3, 0)).

As mentioned above, a template of neighboring positions can be used fordetermining a context for coding a value at a specific scan position ofthe block 1302. The template 1088 of FIG. 10B is again used forillustration purposes. As also mentioned above, any combination of thevalues of the neighboring positions can be used to determine a contextindex (i.e., ctx) that is used to select a context model. Forillustration purposes, the sum combination (i.e., addition) is usedherein.

Super-imposing the template 1088 on each of the scan positions of thescan-order anti-diagonal line 1306 indicates that the templateanti-diagonal line 1202 of FIG. 12 coincides with the scan-orderanti-diagonal line 1308 and the template anti-diagonal line 1204 of FIG.12 coincides with the scan-order anti-diagonal line 1310. As such,coding values (e.g., of the scan positions 49-53) of a currentscan-order anti-diagonal line (e.g., the scan-order anti-diagonal line1306) uses the values of a first scan-order anti-diagonal line (e.g.,the scan-order anti-diagonal line 1308) coded immediately before thecurrent scan-order anti-diagonal line and the values of a secondscan-order anti-diagonal line (e.g., the scan-order anti-diagonal line1310) coded immediately before the first scan-order anti-diagonal line.The scan-order anti-diagonal line 1310 is said to be “older” than thescan-order anti-diagonal line 1308 since its values are coded first.

In the following, the notation {tilde over (X)} (i.e., a number with atilde on top) means “the value of the lower plane at scan location X).For example,

means the value of the lower plane (i.e., the block 1302) at scanlocation 31. That is, and referring to the lower plane array, level, ofFIG. 11, level [31]=

. Additionally, the notation O(X) refers to the context index of thecontext to be used for coding the value at scan position x. O(X) iscalculated using the neighboring values identified in the template usingan operation, such as an addition operation.

Referring to the block 1302, the context indexes of the contexts forcoding the values at scan location 50, 51, and 52 are, respectivelyO(50)=

+

+

+

+

, O(51)=

+

+

+

+

, and O(52)=

+

+

+

+

. If a value is not available, as indicated above with respect to FIGS.10A-10B, it can be assumed to be zero. As such, the context index of thecontext for coding the value at scan location 11 is O(11)=0+{tilde over(9)}+{tilde over (3)}+{tilde over (8)}+{tilde over (4)} since scanlocation 1312 is not available (i.e., the scan location 1312 is outsidethe bounds of the transform block of the block 1302).

As can be seen from O(5), O(51) and O(52), the values used fordetermining the context models of two consecutive positions in ascan-order anti-diagonal line overlap. For example, one value (e.g.,

) from the scan-order anti-diagonal line 1308 and two values (e.g.,

and

) from the scan-order anti-diagonal line 1310 are re-used fordetermining the context indexes O(50) and O(51); and one value (e.g.,

) from the scan-order anti-diagonal line 1308 and two values (e.g.,

and

) from the scan-order anti-diagonal line 1310 are re-used fordetermining the context indexes O(51) and O(52). This pattern isrepeated for all values of a scan-order diagonal line.

To reiterate, two consecutive scan positions (e.g., scan positions 50and 51) along the same scan-order anti-diagonal line (e.g., thescan-order anti-diagonal line 1306) share previously coded values (e.g.,

,

, and

) for calculating context indexes. The shared values are alongpreviously coded scan-order anti-diagonal lines (e.g., the scan-orderanti-diagonal lines 1308, 1310) corresponding to the templateanti-diagonal lines (e.g., the template anti-diagonal lines 1202, 1204).

Additionally, it is noted that the values of each previously codedscan-order anti-diagonal line, which are required for coding a currentto-be-coded value, are contiguous along the respective scan-orderanti-diagonal line. For example, for determining the context indexO(50), the required values

and

are contiguous along the scan-order anti-diagonal line 1308, and therequired values

,

, and

are contiguous along the scan-order anti-diagonal line 1310.Accordingly, by storing the values of each scan-order anti-diagonal linein a separate line buffer, determining a context index can be performedby accessing contiguous locations in each line buffer thereby improvingcoding performance.

The minimum number of line buffers depends on the number of templateanti-diagonal lines. For example, as the template 1088 includes twotemplate anti-diagonal lines (i.e., template anti-diagonal lines 1202,1204), two line buffers need be maintained. For example, if the template1070 of FIG. 10A is used, then three line buffers need be maintained.

For a transform block of size N×N, where M>N, each line buffer caninclude a maximum of N values. As such, for the block 1302 (which is ofsize 8×8), and using the template 1088, two line buffers each of size 8can be used. Line buffer LB1 can correspond to the scan-orderanti-diagonal line 1310. Line buffer LB2 can correspond to thescan-order anti-diagonal line 1308.

The total number of scan-order anti-diagonal lines for a lower plane ofsize N×N is 2N−1. For example, the block 1302 includes 2*8−1=15scan-order anti-diagonal lines. In an implementation, each scan-orderanti-diagonal line can have an index L, where L=0, 1, . . . , 2N−2. Thenumber of values in a scan-order anti-diagonal line L, NUM(L), can begiven by:

${{NUM}(L)} = \left\{ \begin{matrix}{L + 1} & {{{if}\mspace{14mu} L} < N} \\{{2N} - L - 1} & {{{if}\mspace{14mu} L} \geq N}\end{matrix} \right.$

For example, a scan-order anti-diagonal 1314, with an index L=0 and,includes (L+1)=(0+1)=1 value. The scan-order anti-diagonal line 1310 hasan index of L=8. Accordingly, the scan-order anti-diagonal line 1310includes (2*8−8−1)=7 values.

As some of the scan-order anti-diagonal lines may not include N numberof values, some locations of the corresponding line buffer can beassumed to be zero. The number of assumed-to-be-zero locations of theline buffer is equal to (N-NUM(L)). For example, as the scan-orderanti-diagonal line 1308 has an index of L=9, a corresponding lineincludes 2 (i.e., 8-NUM(9)=8−(2*8−9−1)=8−6=2) assumed-to-be-zerolocations.

In an implementation, the assumed-to-be-zero locations of a line buffercan be assumed to be at the head of the line buffer. In anotherimplementation, the assumed-to-be-zero locations can be at the tail ofthe line buffer.

In an implementation, the values in the line buffers can be arrangedwith reference to an edge of the transform block. For example, thebottom edge can be used as the reference. That is, values of ascan-order anti-diagonal line that are closer to the reference edge(e.g., bottom edge) are stored toward the front of the life buffer. Insuch a case, if a scan-order anti-diagonal line visits locations fromthe top of the block towards the bottom of the block, then the scanlocations of the scan-order anti-diagonal line are arranged, in thecorresponding line buffer, in the reverse of the traversal order; and ifa scan-order anti-diagonal line visits locations from the bottom of theblock towards the top of the block, then the scan locations of thescan-order anti-diagonal line are arranged, in the corresponding linebuffer, in the traversal order.

Accordingly, the line buffer LB1 can be arranged as LB1=[0,

,

,

,

,

,

,

] and the line buffer LB2 can be arranged as LB2=[0, 0,

,

,

,

,

,

]. The values of the current scan-order anti-diagonal line can also bearranged in a line buffer, CURR_LB. Accordingly, a CURR_LB can beassociated with the scan-order anti-diagonal line 1306. After coding allthe values of the scan-order anti-diagonal line 1306, CURR_LB can bearranged as CURR_LB=[0, 0, 0,

,

,

,

,

]. As such, the value at scan position x can be found at index y inCURR_LB. For example, the value at scan position 49 can be found atCURR_LB[3], the value at scan position 53 can be found at CURR_LB[7],and so on.

Calculating the context index O(x) of the context to be used for codingthe value at scan position x where the scan position x corresponds tolocation y in the CURR_LB can be performed using the formula:

O(CURR_LB[y])=(LB2[y−1]+LB2[y])+(LB1[y−2]+LB1[y−1]+LB1[y])

That is, determining the context index for a value at scan location yuses: 1) the values at scan positions (y−1) and (y) of the line buffercorresponding to the most recently coded scan-order anti-diagonal lineand 2) the values at scan positions (y−2), (y−1), and (y) of the linebuffer corresponding to the oldest coded scan-order anti-diagonal line.As such, only contiguous location in each line buffer are accessed.While one arrangement of values in line buffers and an indexing schemeare described, a person skilled in art can appreciate that otherarrangements and indexing schemes are possible.

As indicated above, and when using the template 1088, the context modelof a value of a scan-order anti-diagonal line depends on the previoustwo scan-order anti-diagonal lines. As such, caching data (i.e., valuesof the lower plane) for context reference modeling can be localized totwo scan-order anti-diagonal lines, each of size N, rather than beingscattered, for example, over the entire N×N transform block.

FIG. 14 is flowchart diagram of a process 1400 for cache management forcontext selection according to implementations of this disclosure. Theprocess 1400 can be implemented, for example, as a software program thatmay be executed by computing devices such as transmitting station 102 orreceiving station 106. The software program can include machine-readableinstructions that may be stored in a memory such as the memory 204 orthe secondary storage 214, and that, when executed by a processor, suchas CPU 202, may cause the computing device to perform the process 1400.The process 1400 may be implemented in whole or in part in the entropyencoding stage 408 of the encoder 400 and/or entropy decoding stage 502of the decoder 5001. The process 1400 can be implemented usingspecialized hardware or firmware. Multiple processors, memories, orboth, may be used.

The process 1400 can be used when coding (i.e., encoding or decoding) atransform block. The transform block can be of size N×N. The process1400 can be used with a transform block coding process that codes atransform block using a lower plane and a higher map. Specifically, theprocess 1400 can be used in the coding of the lower plane. The lowerplane and the higher map can be as described above.

At 1402, the process 1400 maintains line buffers for scan-orderanti-diagonal lines. Maintain can mean generate, calculate, store,determine, allocate, or maintain in any other way. The numbers of linebuffers maintained by the process 1400 can depend on the number oftemplate anti-diagonal lines. For example, the process 1400 canmaintain, for each of the template anti-diagonal lines, a respectiveline buffer. For example, if the template used is a template such as thetemplate 1088 of FIG. 10B, then the process 1400 can maintain two linebuffers, LB1 and LB2 as described above. Each of the line buffers can beof size N. In describing the process 1400, the template 1088 is used.However, a template of different size or having more than two templateanti-diagonal lines can be used.

At 1404, the process 1400 stores in LB1 first coded anti-diagonalcoefficient levels. The anti-diagonal coefficient levels can be thevalues of the scan-order anti-diagonal line as described with respect toFIG. 13. For example, the process 1400 stores in LB1 the values of thescan-order anti-diagonal line 1310, as described above.

At 1406, the process 1400 stores in LB2 second coded anti-diagonalcoefficient levels. For example, the process 1400 stores in LB2 thevalues of the scan-order anti-diagonal line 1308, as described above.

At 1408, the process 1400 references LB1 and LB2 to compute a contextindex for each coefficient level in a third anti-diagonal line. Forexample, the third anti-diagonal line can be the scan-orderanti-diagonal line 1306. The process 1400 references LB1 and LB2 asdescribed above. The first line buffer LB1 can correspond to the firstscan-order anti-diagonal line (e.g., the scan-order anti-diagonal line1310) that is farther than the second scan-order anti-diagonal line(e.g., the scan-order anti-diagonal line 1308) to the thirdanti-diagonal line (i.e., the current scan-order anti-diagonal linebeing coded, such as the scan-order anti-diagonal line 1306).

At 1410, the process 1400 determines whether there are more scan-orderanti-diagonal lines to code. Alternatively, the process 1400, at 1410,determines whether there are mode lower plane values to code ortransform coefficients to code. If so, the process 1400 proceeds to1412; otherwise, the process 1400 ends at 1414.

At 1412, the process 1400 replaces LB1 with the third anti-diagonalline. When all the coefficient levels of the third anti-diagonal lineare coded, the values stored in LB1 are no longer needed. That is, thevalues stored in LB1 are not used to determine a context index forcoding any other value after the third anti-diagonal line is coded.

However, the values of the third anti-diagonal line and the values ofthe second anti-diagonal line are used in determining contexts for thenext anti-diagonal line. For example, after coding the values of thescan-order anti-diagonal line 1306, a codec proceeds to code the valuesof the next scan-order anti-diagonal line, namely the scan-orderanti-diagonal line 1304. In coding the values of the scan-orderanti-diagonal line 1304, the values of the scan-order anti-diagonallines 1306, 1308 are used. As such, the process 1400 retains LB2 andreplaces LB1 with the scan-order anti-diagonal line 1306. That is, theprocess 1400 replaces the oldest-coded scan-order anti-diagonal linewith the last coded scan-order anti-diagonal line.

The process 1400 repeats the above steps. However, in repeating thesteps, LB1 and LB2 are interchanged. That is, before coding a newscan-order anti-diagonal line, the process 1400 replaces the oldest linebuffer (the one of the LB1 and LB2 that was coded first) with the lastcoded (i.e., most recently coded) scan-order anti-diagonal line. Theprocess 1400 repeats until all scan-order anti-diagonal lines of thelower plane are coded.

As described above with respect to O(50) and O(51), determining thecontext model for a positions in a scan-order anti-diagonal lineoverlaps with its nearest neighbors. A Implementations of thisdisclosure can leverage this repeated pattern to more efficientlycalculate context indexes by using a moving window.

FIG. 15 is a flowchart of a process 1500 for using a moving window forcontext selection according to implementations of this disclosure. Theprocess 1500 can leverage the hardware capabilities of a codec toimprove coding performance. The process 1500 interleaves the data of twoline buffers into one destination buffer. The values of the destinationbuffer are used to implement the moving window as described with respectto cumulative sums.

At 1502, the process 1500 stores in a first line buffer, LB1, firstcoded anti-diagonal coefficient levels and stores in a second linebuffer, LB2, second coded anti-diagonal coefficient levels. The linebuffers LB1 and LB2 can be as described above with respect to theprocess 1400.

At 1504, the process 1500 interleaves the value of the first line bufferLB1 and the second line buffer LB2 into a destination buffer, DEST_BUF.For example, assuming that, as described above, LB1=[0,

,

,

,

,

,

,

] and the LB2=[0, 0,

,

,

,

,

,

], then the destination buffer DEST_BUF=[0, 0,

, 0,

,

,

,

,

,

,

,

,

,

,

,

]. More generally, if LB1=[B2, B4, B6, B8, . . . ] and LB2=[B1, B3, B5,B7, . . . ], then DEST_BUF=[B1, B2, B3, B4, B5, B6, B7, B8, . . . ].

Some hardware platforms, such as x86 and ARM, support direct interleaveoperation between two buffers (for example, via the operations unpacklo,unpackhi). Such operations can typically take two line buffers (e.g.,LB1 and LB2) as input, interleave the data of the two lines buffers, andwrite the result into a destination buffer (e.g., DEST_BUF). As such, acodec implemented on such platforms, obtaining the DEST_BUF can beperformed via the hardware instructions.

At 1506, the process 1500 calculates cumulative sums. In animplementation, the cumulative sums can be stored in a cumulative sumsarray, CUM_SUM. The cumulative sums are obtained over the destinationbuffer. The number of cumulative sums can be equal the total number ofvalues in the destination buffer plus 1. The first cumulative sum valueis zero and each subsequent cumulative sum value adds the subsequentvalue from the destination buffer. As such, the values of the cumulativesums array, CUM_SUM, can be calculated using the formula:

CUM_SUM[i+1]=CUM_SUM[i]+DEST_BUF[i], where CUM_SUM[0]=0.

Using the above formula, the cumulative sums array can include thefollowing values:

CUM_SUM[0]=0

CUM_SUM[1]=0+B1

CUM_SUM[2]=0+B1+B2

CUM_SUM[3]=0+B1+B2+B3

CUM_SUM[4]=0+B1+B2+B3+B4

CUM_SUM[9]=0+B1+B2+B3+B4+. . . +B9

At 1508, the process 1500 determines a context index using thecumulative sums. The calculated context index can be used to determinethe context. The context can be determined using a difference of twocumulative sums. Which two cumulative sums are used depends on the sizeof the template. In an case that the template size is tz_size=5, thecontext index of a value at index i of the current scan-orderanti-diagonal line can be calculated using the formula:

Context index[i]=CUM_SUM[tz_size+2(i−1)]−CUM_SUM[2(i−1)]

For example, for a template of size 5, Context Index[1] can be given byCUM_SUM[5]−CUM_SUM[0]; Context Index[2] can be given byCUM_SUM[7]−CUM_SUM[2]; Context Index[3] can be given byCUM_SUM[9]−CUM_SUM[4]; and so on.

FIG. 16 is flowchart diagram of a process 1600 for coding a transformblock of transform coefficients according to an implementation of thisdisclosure. The process 1600 can be implemented, for example, as asoftware program that may be executed by computing devices such astransmitting station 102 or receiving station 106. The software programcan include machine-readable instructions that may be stored in a memorysuch as the memory 204 or the secondary storage 214, and that, whenexecuted by a processor, such as CPU 202, may cause the computing deviceto perform the process 1600. The process 1600 may be implemented inwhole or in part in the entropy encoding stage 408 of the encoder 400and/or the entropy decoding stage 502 of the decoder 500. The process1600 can be implemented using specialized hardware or firmware. Multipleprocessors, memories, or both, may be used.

At 1602, the process 1600 identifies a lower plane. The lower plane canbe as described above with respect to FIGS. 11-13. That is, values ofthe lower plane code an “up-to” value for some of the transformcoefficients of the transform block. In an implementation, the lowerplane includes values for transform coefficients up to the last non-zerocoefficient of the transform block.

In the case of an encoder, create, form, produce, select, construct,determine, specify, generate, or other identify in any mannerwhatsoever. In the case of the encoder, the values of the lower planeare to be encoded in a encoded bitstream, such as the compressedbitstream 420 of FIG. 4. In the case of a decoder, identify can meanidentify the transform block for which the lower plane values are to bedecoded.

At 1604, the process 1600 processes the lower plane according to a scanorder. The scan can be the anti-diagonally aligned scan order asdescribed above. The scan order can be the scan order of the example1300, as described with respect to FIG. 13.

At 1606, the process 1600 selects a template for entropy-coding thevalues of the lower plane. The template indicates, for a to-be-codedvalue of the lower plane, scan positions of already coded values of thelower plane. The scan positions are arranged, in the template, in atleast two template anti-diagonal lines. For example, the template can bethe template 1088, as described with respect to FIG. 12. Accordingly,the template anti-diagonal lines can be the template anti-diagonal lines1202, 1204.

At 1608, the process 1600 determines whether there are more values ofthe lower plane to code. If so, the process proceeds to 1610; otherwisethe process terminates at 1614.

At 1610, the process selects, based on the template anti-diagonal lines,two or more line buffers. As used in this disclosure, “select” means tocreate, form, produce, identify, construct, determine, specify,generate, or other select in any manner whatsoever. Each of the two ormore line buffers corresponds to a respective scan-order anti-diagonalline. For example, as described with respect to FIG. 13, in codingvalues of the scan-order anti-diagonal line 13-6, a first line bufferLB1 corresponding to the scan-order anti-diagonal line 1310 is used anda second line buffer LB2 corresponding to the scan-order anti-diagonalline 1308 are selected.

At 1612, the process 1600 determines a context using the two or moreline buffers. As described above with respect to FIGS. 10A-10B and FIG.13, the process 1600 determines a context index using the neighboringvalues identified by the template. The context index is determined usinga combination of the values. In an implementation, the combination isthe sum of the values. Using the context index, a context is identified,retrieved, accessed, or the like for coding the current value.

At 1614, the process 1600 codes the value using the determined context.After 1614, the process 1600 proceeds back to 1608 to determine whetherthere are additional values to code.

The process 1600 can additionally include an operation for coding ahigher plane including residual coefficients. A maximum level can beassociated with the lower plane and each residual coefficient of thehigher plane corresponding to a respective non-zero transformcoefficient of the transform block having an absolute value exceedingthe maximum level.

In an example of the process 1600, the template includes five scanpositions arranged along two template anti-diagonal lines. For example,the template can be the template 1088 of FIG. 12. A first templateanti-diagonal line (e.g., the template anti-diagonal line 1202) includesat least two of the five scan positions and a second templateanti-diagonal (e.g., the template anti-diagonal line 1204) includes atleast three scan positions. Each of the at least three scan positionsare different from each of the at least two scan positions.

In an example of the process 1600, the template includes seven scanpositions arranged along three template anti-diagonal lines. Forexample, the template can be the template 1070 of FIG. 10A. A firsttemplate anti-diagonal line includes at least two scan positions of theseven scan positions, a second template anti-diagonal line includes atleast three scan positions of the seven scan positions, and a thirdtemplate anti-diagonal line includes at least four scan positions of theseven scan positions. Each of the at least two scan positions, the atleast three scan positions, and the at least four scan positions aredifferent scan positions.

In yet another example, the process 1600 additionally includesmaintaining, for coding values of a current scan-order anti-diagonalline, a first line buffer and a second line buffer. The first linebuffer and the second line buffer can be as described with respect toFIG. 14. As such, the first line buffer can correspond to a firstscan-order anti-diagonal line scanned, using the backward scan order,immediately before the current anti-diagonal line; and the second linebuffer corresponds to a second scan-order anti-diagonal line scanned,using the backward scan order, immediately before the firstanti-diagonal line. The process 1600 also includes, as described withrespect to FIG. 14, replacing, for coding coefficients of an immediatelysubsequent scan-order anti-diagonal line to the current scan-orderanti-diagonal line, the second line buffer with the first line bufferand replace the first line buffer with a line buffer corresponding tothe current anti-diagonal line.

In yet another example, the process 1600 can include interleaving thefirst line buffer and the second line buffer in a destination buffer.The destination buffer can be used for determining the context forcoding the current value. The destination buffer can be as describedwith respect to FIG. 15.

In an implementation of the process 1600, determining the context forcoding the current value using the destination buffer can includecalculating a respective cumulative sum for each location of thedestination buffer, and determining the context using a differencebetween a first cumulative sum and a second cumulative sum. The firstcumulative sum and the second cumulative sum can be selected based on aposition of the current value.

For simplicity of explanation, the processes 600, 900, 1100, 1400, 1500,and 1600 are each depicted and described as a series of blocks, steps,or operations. However, the blocks, steps, or operations in accordancewith this disclosure can occur in various orders and/or concurrently.Additionally, other steps or operations not presented and describedherein may be used. Furthermore, not all illustrated steps or operationsmay be required to implement a technique in accordance with thedisclosed subject matter.

The aspects of encoding and decoding described above illustrate someencoding and decoding techniques. However, it is to be understood thatencoding and decoding, as those terms are used in the claims, could meancompression, decompression, transformation, or any other processing orchange of data.

The words “example” or “implementation” are used herein to mean servingas an example, instance, or illustration. Any aspect or design describedherein as “example” or “implementation” is not necessarily to beconstrued as preferred or advantageous over other aspects or designs.Rather, use of the words “example” or “implementation” is intended topresent concepts in a concrete fashion. As used in this application, theterm “or” is intended to mean an inclusive “or” rather than an exclusive“or”. That is, unless specified otherwise, or clear from context, “Xincludes A or B” is intended to mean any of the natural inclusivepermutations. That is, if X includes A; X includes B; or X includes bothA and B, then “X includes A or B” is satisfied under any of theforegoing instances. In addition, the articles “a” and “an” as used inthis application and the appended claims should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form. Moreover, use of the term “animplementation” or “one implementation” throughout is not intended tomean the same embodiment or implementation unless described as such.

Implementations of transmitting station 102 and/or receiving station 106(and the algorithms, methods, instructions, etc., stored thereon and/orexecuted thereby, including by encoder 400 and decoder 500) can berealized in hardware, software, or any combination thereof. The hardwarecan include, for example, computers, intellectual property (IP) cores,application-specific integrated circuits (ASICs), programmable logicarrays, optical processors, programmable logic controllers, microcode,microcontrollers, servers, microprocessors, digital signal processors orany other suitable circuit. In the claims, the term “processor” shouldbe understood as encompassing any of the foregoing hardware, eithersingly or in combination. The terms “signal” and “data” are usedinterchangeably. Further, portions of transmitting station 102 andreceiving station 106 do not necessarily have to be implemented in thesame manner

Further, in one aspect, for example, transmitting station 102 orreceiving station 106 can be implemented using a general-purposecomputer or general-purpose processor with a computer program that, whenexecuted, carries out any of the respective methods, algorithms and/orinstructions described herein. In addition, or alternatively, forexample, a special purpose computer/processor can be utilized which cancontain other hardware for carrying out any of the methods, algorithms,or instructions described herein.

Transmitting station 102 and receiving station 106 can, for example, beimplemented on computers in a video conferencing system. Alternatively,transmitting station 102 can be implemented on a server and receivingstation 106 can be implemented on a device separate from the server,such as a hand-held communications device. In this instance,transmitting station 102 can encode content using an encoder 400 into anencoded video signal and transmit the encoded video signal to thecommunications device. In turn, the communications device can thendecode the encoded video signal using a decoder 500. Alternatively, thecommunications device can decode content stored locally on thecommunications device, for example, content that was not transmitted bytransmitting station 102. Other transmitting station 102 and receivingstation 106 implementation schemes are available. For example, receivingstation 106 can be a generally stationary personal computer rather thana portable communications device and/or a device including an encoder400 may also include a decoder 500.

Further, all or a portion of implementations of the present disclosurecan take the form of a computer program product accessible from, forexample, a tangible computer-usable or computer-readable medium. Acomputer-usable or computer-readable medium can be any device that can,for example, tangibly contain, store, communicate, or transport theprogram for use by or in connection with any processor. The medium canbe, for example, an electronic, magnetic, optical, electromagnetic, or asemiconductor device. Other suitable mediums are also available.

The above-described embodiments, implementations and aspects have beendescribed in order to allow easy understanding of the present disclosureand do not limit the present disclosure. On the contrary, the disclosureis intended to cover various modifications and equivalent arrangementsincluded within the scope of the appended claims, which scope is to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structure as is permitted under the law.

What is claimed is:
 1. An apparatus for encoding a transform block oftransform coefficients, the apparatus comprising: a processor configuredto: maintain, for encoding current values related to the transformcoefficients a first line buffer and a second line buffer, wherein thecurrent values are arranged along a current scan-order anti-diagonalline of a scan order, wherein the first line buffer includes firstvalues of a first scan-order anti-diagonal line scanned immediatelybefore the current scan-order anti-diagonal line, and wherein the secondline buffer includes second values of a second scan-order anti-diagonalline scanned immediately before the first scan-order anti-diagonal line;interleave the first values of the first line buffer and the secondvalues of the second line buffer in a destination buffer; select, usingthe destination buffer, a probability distribution for coding a currentvalue of the current values; entropy encode, in a compressed bitstream,the current value using the probability distribution; and replace, forcoding values of an immediately subsequent scan-order anti-diagonal lineto the current scan-order anti-diagonal line, one of the second linebuffer or the first line buffer with the current scan-orderanti-diagonal line.
 2. The apparatus of claim 1, wherein to interleavethe first values of the first line buffer and the second values of thesecond line buffer in the destination buffer comprises to: use ahardware instruction to interleave the first values and the secondvalues in the destination buffer.
 3. The apparatus of claim 2, whereinthe hardware instruction is one of unpackhi or unparcklo.
 4. Theapparatus of claim 2, wherein the processor is further configured to:calculate a respective cumulative sum for at least some locations of thedestination buffer.
 5. The apparatus of claim 4, wherein to calculatethe respective cumulative sum for the at least some locations of thedestination buffer comprises to: add a destination buffer value at alocation of the at least some locations of the destination buffer to acumulative sum corresponding to an immediately preceding location of thedestination buffer.
 6. The apparatus of claim 5, wherein to determine,using the destination buffer, the probability distribution for codingthe current value of the current scan-order anti-diagonal line comprisesto: calculate a difference between a first cumulative sum correspondingto a first location of the destination buffer and a second cumulativesum corresponding to a second location of the destination buffer,wherein the first cumulative sum and the second cumulative sum areselected based on a position of the current value; and determine theprobability distribution using the difference.
 7. The apparatus of claim6, wherein a distance between the first location of the destinationbuffer and the second location of the destination buffer is related to anumber of scan positions of the already coded values of a template,wherein the template indicates, for a to-be-coded value, scan positionsof already coded values, the scan positions arranged, in the template,in at least two anti-diagonal lines.
 8. An apparatus for decoding atransform block, the apparatus comprising: a processor configured to:store, in a first line buffer, first values of a first scan-orderdiagonal line scanned immediately before a current scan-order diagonalline of the transform block; store, in a second line buffer, secondvalues of a second scan-order diagonal line scanned immediately beforethe first scan-order diagonal line; interleave the first values of thefirst line buffer and the second values of the second line buffer in adestination buffer; select, using the destination buffer, a probabilitydistribution for coding a current value of the current scan-orderdiagonal line; entropy decode, from a compressed bitstream, the currentvalue using the probability distribution; and replace, for coding valuesof an immediately subsequent scan-order diagonal line to the currentscan-order diagonal line, one of the second line buffer or the firstline buffer with current values of the current scan-order diagonal line.9. The apparatus of claim 8, wherein to interleave the first values ofthe first line buffer and the second values of the second line buffer inthe destination buffer comprises to: use a hardware instruction tointerleave the first values and the second values in the destinationbuffer.
 10. The apparatus of claim 9, wherein the hardware instructionis one of unpackhi or unparcklo.
 11. The apparatus of claim 9, whereinthe processor is further configured to: calculate a respectivecumulative sum for at least some locations of the destination buffer.12. The apparatus of claim 11, wherein to calculate the respectivecumulative sum for the at least some locations of the destination buffercomprises to: add a destination buffer value at a location of the atleast some locations of the destination buffer to a cumulative sumcorresponding to an immediately preceding location of the destinationbuffer.
 13. The apparatus of claim 12, wherein to determine, using thedestination buffer, the probability distribution for coding the currentvalue of the current scan-order diagonal line comprises to: calculate adifference between a first cumulative sum corresponding to a firstlocation of the destination buffer and a second cumulative sumcorresponding to a second location of the destination buffer, whereinthe first cumulative sum and the second cumulative sum are selectedbased on a position of the current value; and determine the probabilitydistribution using the difference.
 14. The apparatus of claim 13,wherein a distance between the first location of the destination bufferand the second location of the destination buffer is related to a numberof scan positions of the already coded values of a template, wherein thetemplate indicates, for a to-be-coded value, scan positions of alreadycoded values, the scan positions arranged, in the template, in at leasttwo anti-diagonal lines.
 15. A method for decoding a transform block,comprising: storing, in a first line buffer, first values of a firstscan-order diagonal line scanned immediately before a current scan-orderdiagonal line of the transform block; storing, in a second line buffer,second values of a second scan-order diagonal line scanned immediatelybefore the first scan-order diagonal line; interleaving the first valuesof the first line buffer and the second values of the second line bufferin a destination buffer; selecting, using the destination buffer, aprobability distribution for coding a current value of the currentscan-order diagonal line; entropy decoding, from a compressed bitstream,the current value using the probability distribution; and replacing, forcoding values of an immediately subsequent scan-order diagonal line tothe current scan-order diagonal line, one of the second line buffer orthe first line buffer with current values of the current scan-orderdiagonal line.
 16. The method of claim 15, wherein interleaving thefirst values of the first line buffer and the second values of thesecond line buffer in the destination buffer comprises: using a hardwareinstruction to interleave the first values and the second values in thedestination buffer.
 17. The method of claim 16, wherein the hardwareinstruction is one of unpackhi or unparcklo.
 18. The method of claim 16,further comprising: calculating a respective cumulative sum for at leastsome locations of the destination buffer.
 19. The method of claim 18,wherein calculating the respective cumulative sum for the at least somelocations of the destination buffer comprises: adding a destinationbuffer value at a location of the at least some locations of thedestination buffer to a cumulative sum corresponding to an immediatelypreceding location of the destination buffer.
 20. The method of claim19, wherein determining, using the destination buffer, the probabilitydistribution for coding the current value of the current scan-orderdiagonal line comprises: calculating a difference between a firstcumulative sum corresponding to a first location of the destinationbuffer and a second cumulative sum corresponding to a second location ofthe destination buffer, wherein the first cumulative sum and the secondcumulative sum are selected based on a position of the current value;and determining the probability distribution using the difference.